Kibana 8.8.0edit

Review the following information about the Kibana 8.8.0 release.

Known issuesedit

Kibana can run out of memory during an upgrade when there are many Fleet agent policies.

Details
Due to a schema version update, during Fleet setup in 8.8.x, all agent policies are being queried and deployed. This action triggers a lot of queries to the Elastic Package Registry (EPR) to fetch integration packages. As a result, there is an increase in Kibana’s resident memory usage (RSS).

Impact
Because the default batch size of 100 for schema version upgrade of Fleet agent policies is too high, this can cause Kibana to run out of memory during an upgrade. For example, we have observed 1GB Kibana instances run out of memory during an upgrade when there were 20 agent policies with 5 integrations in each.

Workaround
Two workaround options are available:

  • Increase the Kibana instance size to 2GB. So far, we are not able to reproduce the issue with 2GB instances.
  • Set xpack.fleet.setup.agentPolicySchemaUpgradeBatchSize to 2 in the kibana.yml and restart the Kibana instance(s).

In 8.9.0, we are addressing this by changing the default batch size to 2.

Failed upgrades to 8.8.0 can cause bootlooping and data loss

Details
The 8.8.0 release splits the .kibana index into multiple saved object indices. If an upgrade to 8.8.0 partially succeeds, but not all the indices are created successfully, Kibana may be unable to successfully complete the upgrade on the next restart.

This can result in a loss of saved objects during the upgrade. This can also leave Kibana in a bootlooping state where it’s unable to start due to write_blocked indices.

Impact
The 8.8.1 release includes in a fix for this problem. Customers affected by a failed 8.8.0 upgrade should contact Elastic support. For more information, see the related issue.

Memory leak in Fleet audit logging.

Details
Fleet introduced audit logging for various CRUD (create, read, update, and delete) operations in version 8.8.0. While audit logging is not enabled by default, we have identified an off-heap memory leak in the implementation of Fleet audit logging that can result in poor Kibana performance, and in some cases Kibana instances being terminated by the OS kernel’s oom-killer. This memory leak can occur even when Kibana audit logging is not explicitly enabled (regardless of whether xpack.security.audit.enabled is set in the kibana.yml settings file).

Impact
The version 8.8.2 release includes in a fix for this problem. If you are using Fleet integrations and Kibana audit logging in version 8.8.0 or 8.8.1, you should upgrade to 8.8.2 or above to obtain the fix.

Monitors in Synthetics may stop running

Details
If Monitor Management was enabled prior to 8.6.0, the API key generated internally will not contain the required permissions. The Synthetics app will attempt to fix this automatically in #155203 when a user with sufficient privileges visits this page for the first time after upgrading to 8.8.0.

Impact
All monitors configured to run on Elastic’s global managed testing infrastructure will stop running until a user with permissions has loaded the Synthetics app.

Network throttling disabled for browser monitors in Synthetics

Details
Network throttling has been temporarily disabled for browser-based Synthetics monitors running on Elastic’s global managed testing infrastructure and private locations. This will be enabled again at some point in the future. We’re providing frequent updates on this issue in this document.

Impact
With network throttling being disabled, your monitors may run more quickly (i.e. have a lower duration) than you observed previously and than when network throttling is enabled again in the future. No monitor configurations have been changed, but the network throttling settings are ignored at the moment.

Alert failures when migrating to 8.8.0 from 8.6 or earlier

Details
If a cluster meets all of the following conditions, its Elastic Security and Observability rules will fail and no actions will be sent:

  • The Elastic Security and Observability rules were created in version 8.6 or earlier releases.
  • There must be an index template (for any index) that isn’t composed of component templates.

The following error messages in the Kibana log occur when Kibana starts or when the rules run:

Error installing component template .alerts-ecs-mappings - Cannot read properties of undefined (reading 'includes')

Error installing common resources for AlertsService. No additional resources will be installed and rule execution may be impacted. - Failure during installation. Cannot read properties of undefined (reading 'includes')

Impact
If you have upgraded to 8.8.0 and your alerting rules fail, upgrade to 8.8.1.

Incorrect attachments are added to cases

Details
When you attach machine learning visualizations, OsQuery, or Indicators of Compromise (IoCs) to a case, each attachment has its own view which renders in the Activity tab. For these attachments, a bug was introduced in 8.8.0:

  1. If you add two different attachments on a case, the view will be the same for both.
  2. If you add one attachment to one case and another to a different case, in the second case you will view the attachment of the first case.

Alerts are not affected.

Impact
There are no mitigations for the first scenario, other than upgrading to 8.8.1. For the second scenario, refreshing the case fixes the issue.

Error when clicking link to Universal Profiling Agent integration

Details
Clicking Manage Universal Profiling agent in Fleet on the Add profiling data page under the Elastic Agent integration tab results in an error loading integration details.

Impact
You can access the Universal Profiling Agent integration by doing the following:

  1. Select Integrations from the left navigation.
  2. Turn on Display beta integrations in the bottom-left corner of the Integrations page.
  3. Search for Universal Profiling and select Universal Profiling Agent.

Breaking changesedit

Breaking changes can prevent your application from optimal operation and performance. Before you upgrade to 8.8.0, review the breaking changes, then mitigate the impact to your application.

Removes legacy project monitor API

Details
The project monitor API for Synthetics in Elastic Observability has been removed. For more information, refer to #155470.

Impact
In 8.8.0 and later, an error appears when you use the project monitor API.

Changes the privileges for alerts and cases

Details
The privileges for attaching alerts to cases has changed. For more information, refer to #147985.

Impact
To attach alerts to cases, you must have Read access to an Observability or Security feature that has alerts and All access to the Cases feature. For detailed information, check Kibana privileges and Configure access to cases.

New alerts will be generated for Synthetics monitors

Details
All monitor status alerts for the Synthetics app that are active at the time of upgrading will be resolved. New alerts will be created automatically. Refer to #157234.

New Synthetics errors for monitors running on private locations

Details
All Synthetics errors that are in progress for private locations at the time the stack is upgrading will be resolved. A new error state will take its place. Refer to #156324.

To review the breaking changes in previous versions, refer to the following:

8.7.0 | 8.6.0 | 8.5.0 | 8.4.0 | 8.3.0 | 8.2.0 | 8.1.0 | 8.0.0 | 8.0.0-rc2 | 8.0.0-rc1 | 8.0.0-beta1 | 8.0.0-alpha2 | 8.0.0-alpha1

Deprecationsedit

The following functionality is deprecated in 8.8.0, and will be removed in 9.0.0. Deprecated functionality does not have an immediate impact on your application, but we strongly recommend you make the necessary updates after you upgrade to 8.8.0.

Deprecates ephemeral Task Manager settings

Details
The following Task Manager settings are deprecated:

  • xpack.task_manager.ephemeral_tasks.enabled
  • xpack.task_manager.ephemeral_tasks.request_capacity
  • xpack.alerting.maxEphemeralActionsPerAlert

For more information, refer to #154275.

Impact
To improve task execution resiliency, remove the deprecated settings from the kibana.yml file. For detailed information, check Task Manager settings in Kibana.

Deprecates monitor schedules

Details
Synthetics now only accepts the following scheduling options for your monitors: every 1, 3, 5, 10, 15, 20, 30, 60, 120, 240 minutes. Other choices that existed previously have been deprecated. For more information, refer to #154010 and #154952.

Impact
Monitors that were using a scheduling option that is no longer valid have been automatically updated to use the nearest value from the new list of options.

Deprecates Agent reassign API PUT endpoint

Details
The PUT endpoint for the agent reassign API is deprecated. For more information, refer to #152236.

Impact
Use the POST endpoint for the agent reassign API.

Deprecates total in /agent_status Fleet API

Details
The total field in /agent_status Fleet API responses is deprecated. For more information, refer to #151564.

Impact
The /agent_status Fleet API now returns the following statuses:

  • all — All active and inactive
  • active — All active
Deprecates Elastic Synthetics integration

Details
The Elastic Synthetics integration is deprecated. For more information, refer to #149506.

Impact
To monitor endpoints, pages, and user journeys, go to ObservabilitySynthetics (beta).

Featuresedit

Kibana 8.8.0 adds the following new and notable features.

Alerting
  • Adds Maintenance Window Task Runner Integration + New AAD/Event Log Fields #154761
  • Adds support for users authenticated with API keys to manage alerting rules #154189
  • Adds the ability to control allowed attached file mime types and the maximum file size #154013
  • Adds query and timeframe params to RuleAction to filter alerts #152360
APM
  • Adds group-by feature in APM rules #155001
  • Adds queues as nodes to the service map #153784
  • Adds the ability to display the latest agent version in agent explorer #153643
  • Adds table tabs showing summary of metrics #153044
  • Adds warning to Edit Rule Flyout when publicUrl is not configured #149832
Cases
  • Adds support for file attachments in Cases #154436
  • Adds the Cases column to the alerts table #150963
  • Adds filtering and sorting for the case activity #149396
  • Adds the ability to filter user activities with pagination #152702
Dashboard
Pins the unified search bar and dashboard toolbar to the top of the dashboard page when scrolling #145628
Discover
Adds log pattern analysis #153449
Elastic Security
For the Elastic Security 8.8.0 release information, refer to Elastic Security Solution Release Notes.
Enterprise Search
For the Elastic Enterprise Search 8.8.0 release information, refer to Elastic Enterprise Search Documentation Release notes.
Fleet
  • Adds audit logging for core CRUD operations #152118
  • Adds modal to display versions changelog #152082
Infrastructure
  • Adds the logs tab to the Hosts View #152995
  • Adds Alerts tab into Hosts View #149579
  • Adds refactoring to the Time and Position log stream state #149052
Machine Learning
  • Adds ELSER config to the Trained Models UI #155867
  • Adds support for custom URLs in jobs for Data Frame Analytics #154287
  • Adds support to filter fields from grouping in Explain Log Rate Spikes #153864
  • Adds log pattern analysis in Discover #153449
Management
  • Adds support for global settings #148975
  • Adds Custom Branding settings to Global settings #150080
Maps
Adds map.emsUrl to docker env variables #153441
Observability
  • Adds the ability to changes all SLO assets to managed, and indices to hidden #154953
  • Adds Exploratory View to a separate app #153852
Platform
Adds text #151631
Security
  • Adds CloudFormation agent install method #155045
  • Adds Vul mgmt flyout details panel #154873
  • Adds Vulnerabilities Table #154388
  • Adds the ability to select a theme preference for Kibana in the User Profile #151507
Uptime
Adds UUID to RuleAction #148038

For more information about the features introduced in 8.8.0, refer to What’s new in 8.8.