Load balancingedit

The template exposes three different load balancing deployment options to suit different use cases. Each is exposed through

loadBalancerType
Choose between internal, external or gateway to configure an internal load balancer, external load balancer, or Application Gateway, respectively. Default is internal.

Internal load balanceredit

An OSI layer 4 load balancer configured with a static private IP address on the virtual network, that can be used to round robin requests across Elasticsearch nodes. The private IP address is configured with vNetLoadBalancerIp parameter.

The following settings apply to the internal load balancer

loadBalancerInternalSku
Choose between Basic and Standard load balancer SKUs for the internal load balancer. An internal load balancer is always deployed, to balance internal traffic to the cluster. When the Standard load balancer SKU is selected and loadBalancerType is internal, a Network Security Group will also be deployed, and each VM in the backend pool will be assigned a Standard public IP address, to allow outbound internet traffic from the VMs in the backend pool, which is required to install the Elastic Stack and dependencies. Check the Azure documentation on Standard Load Balancers to determine which options is best suited for your needs. Default is Basic.

When coordinating nodes are deployed, they are attached to the internal load balancer backend pool. When no coordinating nodes are deployed, the data nodes are attached to the backend pool. The load balancer receives incoming requests on port 9200 and round robins them across the backend pool over port 9200, with a TCP health probe that checks connectivity every 30 seconds, taking nodes out of the backend pool when health probes fail. An idle timeout of 5 minutes is also configured.

When Kibana is deployed, Kibana is configured to communicate with Elasticsearch through the internal load balancer.

External load balanceredit

An OSI layer 4 load balancer configured with a public IP address that can be used to send requests to Elasticsearch from the public internet.

The following settings are applicable to the external load balancer

loadBalancerExternalSku
Choose between Basic and Standard load balancer SKUs for the external load balancer. Only relevant when loadBalancerType is external. When the Standard load balancer SKU is selected, the public IP address SKU attached to the external load balancer will also be Standard, and a Network Security Group will also be deployed, to allow inbound internet traffic to the VMs in the backend pool. Check the Azure documentation on Standard Load Balancers to determine which options is best suited for your needs. Default is Basic.

When choosing to deploy with an external load balancer, both an internal and external load balancer are deployed; when Kibana is deployed, it will be configured to communicate with Elasticsearch through the internal load balancer. The external load balancer receives incoming requests on port 9200 and round robins them across the backend pool over port 9201, with a TCP health probe that checks connectivity every 30 seconds, taking nodes out of the backend pool when health probes fail. Since both the internal and external load balancer contain the same VMs in their backend pools, the external load balancer communicates with the backend pool over port 9201, to work around a limitation in Azure. On each Elasticsearch node VM, incoming traffic on port 9201 is re-routed to port 9200 using a persistent iptables configuration, and then routed back to 9201 for the outgoing response.

As with the internal load balancer, when coordinating nodes are deployed, they are attached to the external load balancer backend pool. When no coordinating nodes are deployed, data nodes are attached to the backend pool. The load balancer round robins requests across the backend pool over port 9200, with a TCP health probe that checks connectivity every 30 seconds, taking nodes out of the backend pool when health probes fail. An idle timeout of 5 minutes is also configured.

Application Gatewayedit

An OSI layer 7 load balancer configured with a dynamically assigned public IP address that can be used to send requests to Elasticsearch from the public internet. Application Gateway can be configured with a certificate to configure SSL/TLS between the client and Application Gateway, performing SSL termination at the gateway. The following parameters are used to configure Application Gateway

vNetAppGatewaySubnetName
The subnet name for Application Gateway. Application Gateway can only be deployed into a subnet that contains Application Gateways. When deploying to an existing virtual network, the subnet must exist beforehand. Defaults to es-gateway-subnet.
vNetNewAppGatewaySubnetAddressPrefix
The address space of the Application Gateway subnet. This must be specified when deploying a new virtual network. The address space must be large enough for the specified Application Gateway configuration; see Application Gateway FAQs for more details
appGatewayTier
The tier of Application Gateway. Either Standard or WAF. The latter is Web application firewall (WAF), a feature that provides centralized protection from common exploits and vulnerabilities based on OWASP core rule sets 3.0. Defaults to Standard.
appGatewaySku
The size of the Application Gateway. Choose Small, Medium or Large. When choosing appGatewayTier of WAF, the size must be at least Medium. The main difference between sizes is the amount of throughput that the Gateway can handle. Defaults to Medium.
appGatewayCount
The number of instances of Application Gateway, from 1 to 10. a minimum value of 2 is recommended for production loads. Defaults to 2.
appGatewayCertBlob
A Base-64 encoded PKCS#12 archive (.p12/.pfx) containing the certificate and key for Application Gateway. This certificate is used to configure SSL/TLS to and from Application Gateway. This is required to correctly configure Application Gateway.
appGatewayCertPassword
The passphrase for the PKCS#12 archive containing the certificate and key for Application Gateway. Defaults to empty string.
appGatewayEsHttpCertBlob
The Base-64 encoded public certificate (.cer) used to secure the HTTP layer of Elasticsearch. This is used by Application Gateway to whitelist certificates used by the backend pool when configuring end-to-end encryption, and is required when SSL/TLS is configured for the HTTP layer of Elasticsearch. See the Transport Layer Security section for more details.
appGatewayWafStatus
When choosing appGatewayTier of WAF, this parameter controls the firewall status, either Enabled or Disabled. Defaults to Enabled.
appGatewayWafMode
When choosing appGatewayTier of WAF, the firewall mode, either Detection or Prevention. Defaults to Detection.

When choosing to deploy with Application Gateway, both Application Gateway and an internal load balancer are deployed; the internal load balancer is configured as the backend pool for Application Gateway.

A custom health probe is configured that reports healthy for the backend pool for status codes between 200-399, and for status code 401, which may be returned when Elastic Stack Security is enabled, since the health probe makes requests without any form of authentication.

The public IP address of Application Gateway is not returned in the template outputs, but can be retrieved after successful deployment with

Azure CLI 2.0.

gateway_ip_resource=$(az network public-ip show --name "es-app-gateway-ip" \
  --resource-group "<resource group>")

# jq needs to be installed
gateway_fqdn=$(jq -r .dnsSettings.fqdn <<< $gateway_ip_resource)
gateway_ip="https://${gateway_fqdn}:9200"

Azure PowerShell.

$gatewayIpResource = Get-AzureRmPublicIpAddress -Name "es-app-gateway-ip" `
  -ResourceGroupName "<resource group>"

$gatewayIp = "https://$($gatewayIpResource.DnsSettings.Fqdn):9200"