New security setting, in-place configuration changes, new hardware support, and signup with Google releasededit
Elasticsearch Service now supports additional features that let you do more with our hosted offering:
Support for new security setting. The
xpack.http.ssl.cipher_suitessetting is now enabled for Elasticsearch Service. This setting can be useful when using outgoing TLS traffic to external services with restrictive supported cipher suites. One example would be using Telegram from Watcher.
In-place configuration changes. In-place changes for existing deployments substantially speed up configuration changes and are now the default for most operations that involve settings changes, upgrades, and deployment resizing. The benefits include:
- Reduced wait times
- Reduced chance of plan failures and improved platform stability
- Reduced data transfer charges from the cloud platform provider, proportional to the amount of data you manage in your deployment
The speed and reliability of in-place configuration changes comes from applying the changes to the existing instances of your deployment, such as the Elasticsearch nodes and Kibana instances, followed by a rolling restart. There is no downtime for highly available deployments with two or more availability zones, provided that deployments are sized to support their workloads.
Elasticsearch Service previously used only a grow-and-shrink approach for deployment changes. This approach adds new instances, migrates data from old instances to the new ones, and then shrinks the deployment by removing the old instances. Grow-and-shrink provides high availability during configuration changes even for single availability zones. The tradeoff is that changes frequently require replacing some or all of the deployment, triggering a potentially long-running data migration process for larger deployments.
When you change your deployment configuration, Elasticsearch Service will choose the right approach to apply the changes, using either the preferred in-place approach where possible or falling back to grow-and-shrink when necessary. Deployment changes that will continue to require a grow-and-shrink approach include:
- Configuration changes and upgrades to non-highly available deployments that use only one availability zone
- Changes to APM and Kibana instances
When applying changes to non-highly available deployments, the Elasticsearch Service Console now warns you that your deployment may suffer a temporary loss of availability. If loss of availability is not acceptable, make sure to use at least two availability zones, even if only temporarily. Keep in mind that production systems should never use only one availability zone.
The grow-and-shrink approach may still be used in some other scenarios, for example when the host that your deployment runs on has insufficient capacity to scale up an Elasticsearch cluster and needs to use capacity on another host.
New deployment templates to support AWS M5d and R5d hardware. AWS provides memory- and CPU-optimised machines with attached NVMe storage instead of GP2 which offers more performant hardware with better disk performance. In order to support the new hardware, you can now make use of updated AWS deployment templates. The new templates still use the same naming convention and replace components using R4 with R5d, and M5 with M5d.
What’s moving from R4 to R5d:
- Master nodes
- Memory-optimised data nodes
What’s moving from M5 to M5d:
- Coordinating nodes
- Machine learning nodes
- CPU-optimized data nodes
- App Search and Enterprise Search components
Prices for the new hardware are marginally higher. You can check the pricing calculator for exact prices.
New deployments will use the AWS M5d and R5d hardware by default. For existing deployments, there is currently no direct migration path, but you can make use of the new hardware by creating a new deployment and then restoring your data from a snapshot.
Social signup with Google. New users can now use social signup with Google, eliminating the need to verify email addresses or manage passwords when signing up for Elasticsearch Service.
How it works:
- Google social signup is available to all new Elastic Cloud trial and training users.
- GCP Marketplace and AWS Marketplace users still need to sign up with their email address and password just as before. We hope to offer social signup for marketplace users at a later date.
- GovCloud and Elasticsearch Service Private environments do not support social signup.
This version includes the following breaking changes:
Elasticsearch clusters provisioned prior to January 25th, 2019 have a CORS policy set at the proxy layer. This policy is now deprecated and we will be moving to a model that requires you to configure your own http module CORS policy. The
http.cors.*settings are configurable as Elasticsearch user settings for all of your deployments.
If your use case depends on the ability to receive CORS requests and you have an Elasticsearch cluster that was provisioned prior to January 25th 2019, you must manually apply the
http.cors.*settings before the September 22 deadline, or you will potentially have an operational impact.
The in-place configuration change improvements included in this release mean that single availability zone deployments will be temporarily unavailable while the change is applied. If availability is a concern, you should use two availability zones for your deployment, which requires no downtime and which ensures deployment availability during future configuration changes, upgrades, and hardware failures.
As a best practice for production setups, you should never run with a single availability zone if downtime is a concern.
- Traffic filters must not be enabled on monitoring clusters. If you are shipping your logs or metrics to a monitoring cluster, do not associate traffic filters with the Elasticsearch cluster you are sending the logs or metrics to. Since you cannot explicitly allow traffic from certain clusters currently, associating traffic filters with your monitoring cluster will result in clusters being blocked when sending data.
- A known bug in Elasticsearch affects clusters with versions 7.7.0 and 7.7.1 and can prevent you from making configuration changes. If this problem occurs, we recommend that you retry the configuration change. If retrying the change fails please contact Support. We also recommend that you to upgrade to version 7.8 or higher, where the problem has been addressed.
- Configurable limits in Enterprise Search are not supported on Cloud but will be in a future release.
Service release: June 17, 2020
Intro to Kibana
ELK for Logs & Metrics