Elasticsearch Add-On for Heroku hardware

A banner showing a plane trailing several stylized UI sliders superimposed over cloud hardware

Use this information to learn how Elasticsearch Add-On for Heroku instance configurations such as gcp.kibana.1 or azure.data.highio.l32sv2 relate to the underlying cloud platform hardware that we use when you create an Elasticsearch Add-On for Heroku deployment.

Instance configurations

Deployments use a range of virtualized hardware resources from a cloud provider, such as Amazon EC2 (AWS), Google Compute Platform (GCP) or Microsoft Azure. We call these virtualized hardware resources instance configurations on Elasticsearch Add-On for Heroku, but you might recognize the underlying hardware they build on as instance types on AWS and Azure, or custom machine types on GCP.

Instance configurations enable the products and features of the Elastic Stack to run on suitable virtualized hardware resources that support their intended purpose. For example: If you have a logging use case that benefits from Elasticsearch data nodes with large amounts of slower but more cost-efficient storage space, your instance configuration will use large spindle drives rather than more expensive SSD storage. Each instance configuration provides a specific combination of CPU resources, memory, and storage, all of which you can scale from small to very large.

How to read an Elasticsearch Add-On for Heroku instance configuration name, such as aws.data.highio.i3:

CLOUD_PLATFORM.ELASTIC_CLOUD_INSTANCE_TYPE.HARDWARE_PROFILE.HARDWARE_TYPE
CLOUD_PLATFORM
The cloud platform that the underlying hardware belongs to, such as Amazon EC2 (AWS), Google Compute Platform (GCP) or Azure.
ELASTIC_CLOUD_INSTANCE_TYPE

The products or features of the Elastic Stack that can be hosted on the instance configuration, including:

  • Elasticsearch master, ingest, and data nodes
  • Kibana instances
  • Machine learning (ML) nodes
  • Application performance monitoring (APM) server instances
HARDWARE_PROFILE
Optional: The hardware resource characteristics of the instance configuration, such as optimization for I/O, storage, CPU (compute), or memory.
HARDWARE_TYPE
The short name for an AWS or Azure instance type, or a number for a custom GCP machine type on which Elasticsearch Add-On for Heroku hosts an instance configuration. Host machines are shared between deployments, but containerization and guaranteed resource assignment for each deployment prevent a noisy neighbor effect.

A word about terminology: Elasticsearch Add-On for Heroku instance configurations are not the same AWS instance types. Elasticsearch Add-On for Heroku instances configurations refer to the deployable products or features of the Elastic Stack that we support as part of a deployment template, such as Elasticsearch and Kibana. Sometimes, you can also see them referred to as instances, which just means that they are running in a deployment. When you resize or change the configuration of Elasticsearch, for example, you are working with the associated instances, in this case Elasticsearch nodes.

Amazon EC2 (AWS)

Instance configurations map to AWS EC2 instance types as follows:

Instance configurationInstance typesAWS EC2 instance typeMemory Sizes1

aws.data.highio.i3

Elasticsearch data nodes optimized for balanced RAM/vCPU/Disk ratios and performance

i3.8xl2

1, 2, 4, 8, 15, 29, 58, 116, 174, [+58* n] …​

aws.data.highstorage.d2

Elasticsearch data nodes optimized for cost effective storage

d2.4xl3

2, 4, 8, 15, 29, 58, 116, 174, [+58 * n] …​

aws.data.highcpu.m5

Elasticsearch data nodes optimized for high CPU performance 1:2 vCPU allocation compared to highio types

m5.12xl4

1, 2, 4, 8, 15, 30, 60, 120, 180, [+60 * n] …​

aws.data.highmem.r4

Elasticsearch data nodes optimized for lower cost with lower storage ratio

r4.8xl5

1, 2, 4, 8, 15, 29, 58, 116, 174, [+58 * n] …​

aws.master.r4

Elasticsearch master eligible nodes used as tie-breakers to establish a quorum in case of 2 availability zone deployments, or as dedicated master across 3 availability zones

r4.8xl5

1, 2, 4, 8, 15

aws.ingest.m5

Ingest nodes

m5.12xl4

…​

aws.ml.m5

Data nodes

m5.12xl4

1, 2, 4, 8, 15, 30, 60, 120, 180, [+60 * n] …​

aws.kibana.r4

Kibana

r4.8xl

1, 2, 4, 8, 16, 24, [+8 * n] …​

aws.apm.r4

APM

r4.8xl5

0.5, 1, 2, 4, 8, 16, 24, [+8 * n] …​

aws.ccs.r4

Cross-cluster search

r4.8xl

1, 2, 4, 8, [+8 * n] …​

1 Memory sizes ensure efficient hardware utilization and might not scale to the power of two (n2). For sizes above 58, 60 or 64 GB (depending on instance type), we create multiple instances or nodes to ensure efficient JVM heap sizes. For example: If you provision a deployment with a 128 GB Elasticsearch cluster, two 64 GB nodes get created. To learn more about why we offer these JVM heap sizes, see Heap: Sizing and Swapping.

2 NVMe SSD storage (memory:storage ratio of 1:30)

3 HDD-based local storage (memory:storage ratio of 1:100)

4 EBS GP2 storage 1024 GB (memory:storage ratio of 1:8)

5 EBS GP2 storage 1024 GB (memory:storage ratio of 1:2)

To learn more about AWS EC2 hardware instance types, see Amazon EC2 Instance Types.

Google Cloud Platform (GCP)

Instance configurations map to instance types and GCP machine types as follows. Note that Elasticsearch Add-On for Heroku uses custom GCP machine types.

Instance configurationInstance typesGCP machine typeMemory Sizes1

gcp.data.highio.1

Elasticsearch data nodes optimized for balanced RAM/vCPU/Disk ratios and performance

12 cores, 78 GB memory, 3000 GB storage

1, 2, 4, 8, 16, 32, 64, 128, [+64 * n] …​

gcp.data.highstorage.1

Elasticsearch data nodes optimized for cost effective storage

22 cores, 140 GB memory, 12800 GB storage2

2, 4, 8, 16, 32, 64, 128, [+64 * n] …​

gcp.data.highcpu.1

Elasticsearch data nodes optimized for high CPU performance 1:2 vCPU allocation compared to highio types

64 cores, 270 GB memory, 1024 GB storage3

1, 2, 4, 8, 16, 32, 64, 128, [+64 * n] …​

gcp.data.highmem.1

Elasticsearch data nodes optimized for lower cost with lower storage ratio

22 cores, 140 GB memory, 1024 GB storage4

1, 2, 4, 8, 16, 32, 64, 128, [+64 * n] …​

gcp.master.1

Elasticsearch master eligible nodes used as tie-breakers to establish a quorum in case of 2 availability zone deployments, or as dedicated master across 3 availability zones

22 cores, 140 GB memory, 1024 GB storage4

1, 2, 4, 8, 16, …​

gcp.ingest.1

Ingest nodes

64 cores, 270 GB memory, 1024 GB storage4

…​

gcp.ml.1

ML

64 cores, 270 GB memory, 1024 GB storage3

1, 2, 4, 8, 16, 32, 64, 128, [+64 * n] …​

gcp.kibana.1

Kibana

22 cores, 140 GB memory, 1024 GB storage4

1, 2, 4, 8, 16, [+8 * n] …​

gcp.apm.1

APM

22 cores, 140 GB memory, 1024 GB storage4

1, 2, 4, 8, 16, [+8 * n] …​

gcp.ccs.1

Cross-cluster search

22 cores, 140 GB memory, 1024 GB storage

1, 2, 4, 8, 16, [+8 * n] …​

1 Memory sizes ensure efficient hardware utilization and might not scale to the power of two (n2). For sizes above 64 GB, we create multiple instances or nodes to ensure efficient JVM heap sizes. For example: If you provision a deployment with a 128 GB Elasticsearch cluster, two 64 GB nodes get created. To learn more about why we offer these JVM heap sizes, see Heap: Sizing and Swapping.

2 Using SSD persistent volumes, no RAID (memory:storage ratio of 1:100)

3 Using local SSD, no RAID (memory:storage ratio of 1:8)

4 Using local SSD, no RAID (memory:storage ratio of 1:2)

To learn more about GCP hardware machine types, see Machine Types.

Azure

Azure is currently supported for Elasticsearch Add-On for Heroku version 7.3 and 6.8.3. Future versions of the Elastic stack will be made available on Azure regions on the same day they’re released, just like regions in other cloud providers supported by ESS.

Instance configurations map to Azure instance types as follows:

Instance configurationInstance typesAzure instance typeMemory Sizes1

azure.data.highio.l32sv2

Elasticsearch data nodes optimized for balanced RAM/vCPU/Disk ratios and performance

L32sv22

1, 2, 4, 8, 15, 29, 58, 116, 174, [+58* n]​

azure.data.highstorage.e16sv3

Elasticsearch data nodes optimized for cost effective storage

E16sv33

2, 4, 8, 15, 29, 58, 116, 174,[+58 * n]

azure.data.highcpu.d64sv3

Elasticsearch data nodes optimized for high CPU performance 1:2 vCPU allocation compared to highio types

D64sv34

1, 2, 4, 8, 15, 30, 60, 120, 180, [+60 * n]

azure.data.highmem.e32sv3

Elasticsearch data nodes optimized for lower cost with lower storage ratio

E16sv35

1, 2, 4, 8, 15, 29, 58, 116, 174, [+58 * n] …​​

azure.master.e32sv3

Master nodes

E32vs3

1, 2, 4, 8, 15

azure.ccs.e32sv3

Cross-cluster search

E32sv3

1, 2, 4, 8, [+8 * n], …​

azure.ml.d64sv3

ML

D64sv3

1, 2, 4, 8, 15, 30, 60, 120, 180, [+60 * n]

azure.ingest.d64sv3

Ingest nodes

D64sv3

…​

azure.kibana.e32sv3

Kibana

E32sv3

1, 2, 4, 8, 16, 24, [+8 * n], …​

azure.apm.e32sv3

APM

E32sv3

0.5, 1, 2, 4, 8, 16, 24, [+8 * n], …​

1 Memory sizes ensure efficient hardware utilization and might not scale to the power of two (n2). For sizes above 64 GB, we create multiple instances or nodes to ensure efficient JVM heap sizes. For example: If you provision a deployment with a 128 GB Elasticsearch cluster, two 64 GB nodes get created. To learn more about why we offer these JVM heap sizes, see Heap: Sizing and Swapping.

2 Local NVMe SSD storage (memory:storage ratio of 1:30)

3 6xS40 attached storage (memory:storage ratio of 1:100)

4 1xE40 attached storage (memory:storage ratio of 1:8)

5 1xE20 attached storage (memory:storage ratio of 1:2)

To learn more about Azure hardware instance types, see Azure Instance Types.