To help you better understand the available autoscaling settings, this example describes a typical autoscaling workflow on sample Elasticsearch Add-On for Heroku deployment.
- When you upgrade a deployment from a stack version in which autoscaling is not supported (prior to version 7.11) to a version in which it is supported (version 7.11 or higher), after the upgrade completes a message indicates that autoscaling is available. You can enable it simply using the button in the message.
- On a newly created deployment with stack version 7.11 or higher, open the deployment Edit page to find the option to turn on autoscaling.
When you create a new deployment, you can find the autoscaling option under Advanced settings.
Once you confirm your changes or create a new deployment, autoscaling is activated with system default settings that you can adjust as needed (though for most use cases the default settings will likely suffice).
View and adjust autoscaling settings on data tiers:
Open the Edit page for your deployment to get the current and maximum size per zone of each Elasticsearch data tier. In this example, the hot data and content tier has the following settings:
Current size per zone
Maximum size per zone
Up to 2.5 vCPU
The fault tolerance for the data tier is set to 2 availability zones.
- Use the dropdown boxes to adjust the current and/or the maximum size of the data tier. Capacity will be added to the hot content and data tier when required, based on its past and present storage usage, until it reaches the maximum size per zone. Any scaling events are applied simultaneously across availability zones. In this example, the tier has plenty of room to scale relative to its current size, and it will not scale above the maximum size setting. There is no minimum size setting since downward scaling is currently not supported on data tiers.
View and adjust autoscaling settings on a machine learning instance:
From the deployment Edit page you can check the minimum and maximum size of your deployment’s machine learning instances. In this example, the machine learning instance has the following settings:
Minimum size per zone
Maximum size per zone
0.5 vCPU up to 8 vCPU
The fault tolerance for the machine learning instance is set to 1 availability zone.
- Use the dropdown boxes to adjust the minimum and/or the maximum size of the data tier. Capacity will be added to or removed from the machine learning instances as needed. The need for a scaling event is determined by the expected memory and vCPU requirements for the currently configured machine learning job. Any scaling events are applied simultaneously across availability zones. Note that unlike data tiers, machine learning nodes do not have a Current size per zone setting. That setting is not needed since machine learning nodes support both upward and downward scaling.
- Over time, the volume of data and the size of any machine learning jobs in your deployment are likely to grow. Let’s assume that to meet storage requirements your hot data tier has scaled up to its maximum allowed size of 64GB RAM and 32 vCPU. At this point, a notification appears on the deployment overview page letting you know that the tier has scaled to capacity. You’ll also receive an alert by email.
- If you expect a continued increase in either storage, memory, or vCPU requirements, you can use the Maximum size per zone dropdown box to adjust the maximum capacity settings for your data tiers and machine learning instances, as appropriate. And, you can always re-adjust these levels downward if the requirements change.
As you can see, autoscaling greatly reduces the manual work involved to manage a deployment. The deployment capacity adjusts automatically as demands change, within the boundaries that you define. Check our main Deployment autoscaling page for more information.
Intro to Kibana
ELK for Logs & Metrics