Add custom bundles and plugins to your deployment

You can extend your Elasticsearch clusters with custom plugins, and with bundles of external configuration files that Elasticsearch reads at runtime.

You host each ZIP file yourself, on a web server that every allocator in your environment can reach over HTTP or HTTPS, and then reference that URL in your deployment configuration. The file is downloaded each time an Elasticsearch instance starts, so the URL must remain available for as long as your deployment references it.

This page explains how plugins and bundles differ, how to reference either one in your deployment configuration, and then walks through adding a custom plugin and the most common bundles.

Prepare your files

Before you reference a ZIP file, decide whether Elastic Cloud Enterprise should treat it as a plugin or as a bundle. The two are configured separately and behave differently once they reach your Elasticsearch instances:

Plugins

Use a plugin to add functionality to Elasticsearch: an official Elasticsearch plugin that is not provided with Elastic Cloud Enterprise, a community-sourced plugin, or one that you write yourself.

A plugin is a ZIP file that contains a plugin descriptor file and binaries. The descriptor file is called stable-plugin-descriptor.properties for plugins built against the stable plugin API, or plugin-descriptor.properties for plugins built against the classic plugin API. A plugin ZIP file should contain only one descriptor file.

Elasticsearch assumes that the ZIP file contains binaries. If it finds any source code, it fails with an error message, causing provisioning to fail. Make sure the ZIP file contains binaries, and not source code.

Bundles

Use a bundle to make configuration files, such as custom dictionaries, certificates, or SAML metadata, available to every Elasticsearch instance. Bundles are not installed as plugins.

A bundle is a ZIP file whose entire content is extracted to the /app/config directory of every Elasticsearch node. The directory structure inside the ZIP file determines where the files land: folders such as truststore, saml, and ingest-geoip keep their names, so a bundle that contains truststore/keystore.ks makes the keystore available at /app/config/truststore/keystore.ks. Use that full path when you reference the file in your Elasticsearch settings.

Reference your files in your deployment

Host your ZIP file at an HTTP or HTTPS URL that your Elasticsearch instances can reach, then point your deployment configuration at it.

Important

When referencing plugins or bundles, URLs using https with a certificate signed by an internal Certificate Authority (CA) are not supported. Either use a publicly trusted certificate, or fall back to the http scheme.
Avoid using the same URL to serve newer versions of a plugin or bundle, as this may cause different nodes within the same cluster to run different plugin versions. Whenever you update the content of the bundle or plugin, use a new URL in the deployment configuration as well.
If the URL becomes unreachable (if the URL changes at remote end, or connectivity to the remote web server has issues) you might encounter boot loops if Elasticsearch instances are restarted.

To configure custom bundles and plugins to your Elasticsearch clusters and make them available to all Elasticsearch instances, you update your Elasticsearch cluster using the advanced configuration editor:

For bundles, modify the resources.elasticsearch.plan.elasticsearch.user_bundles JSON attribute.
For plugins, modify the resources.elasticsearch.plan.elasticsearch.user_plugins JSON attribute.

Add a custom plugin

Custom plugins can include the official Elasticsearch plugins not provided with Elastic Cloud Enterprise, any of the community-sourced plugins, or plugins that you write yourself.

Log into the Cloud UI.
From the Deployments page, select your deployment.

Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
In the left side navigation select Edit from your deployment menu, then go to the bottom of the page and select Advanced Edit.

Within the Deployment configuration JSON find the section:

resources > elasticsearch > plan > elasticsearch

If there is an existing user_plugins section, then add the new plugin there, otherwise add a user_plugins section.

		{
...
  "resources": {
    "elasticsearch": [
     ...
        "plan": {
         ...
          "elasticsearch": {
            ...
            "user_bundles": [
            {
                ....
            } ] ,
            "user_plugins": [
              {
                "url" : "<some static non_expirable url>",
                "name" : "plugin_name",
                "elasticsearch_version" : "<es_version>"
              },
              {
                "url": "<MY_HOST_URL>/my-custom-plugin.zip",
                "name": "my-custom-plugin",
                "elasticsearch_version": "7.17.1"
              }
            ]
          }
		
	

The URL for the plugin must be always available. Make sure you host the plugin artifacts internally in a highly available environment. The URL must use the scheme http or https
The version must match exactly your Elasticsearch version, such as 7.17.1. Wildcards (*) are not allowed.

Save your changes.
To verify that all nodes have the plugins installed, use one of these commands: GET /_nodes/plugins?filter_path=nodes.*.plugins or GET _cat/plugins?v

Example: Add a custom LDAP bundle

This example adds a custom LDAP bundle for deployment level role-based access control (RBAC). To set platform level RBAC, check Manage users and roles.

Prepare a custom bundle as a ZIP file that contains your keystore file with the private key and certificate inside of a truststore folder. This bundle allows all Elasticsearch containers to access the same keystore file through your ssl.truststore settings.
In the advanced configuration editor, update your new Elasticsearch cluster with the custom bundle you have created. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example:
```
{
...
  "resources": {
    "elasticsearch": [
     ...
        "plan": {
         ...
          "elasticsearch": {
            ...
            "user_bundles": [
            {
              "name": "ldap-cert",
              "url": "<MY_HOST_URL>/ldapcert.zip",
              "elasticsearch_version": "*"
            }
          ]
        }
        ...
		
```
1. The URLs for the bundle ZIP files (ldapcert.zip) must be always available. Make sure you host the plugin artifacts internally in a highly available environment.
Note where the bundle contents are placed, so that you can reference the keystore in your ssl.truststore settings.
```
$ tree .
.
└── truststore
      └── keystore.ks
		
```
In this example, the unzipped keystore file gets placed under /app/config/truststore/keystore.ks.

Example: Add a custom SAML bundle

This example adds a custom SAML bundle for deployment level role-based access control (RBAC). To set platform level RBAC, check Manage users and roles.

In this example, we assume the Identity Provider does not publish its SAML metadata at an HTTP URL, so we provide it through a custom bundle.

Prepare a ZIP file with a custom bundle that contains your Identity Provider’s metadata (metadata.xml). Place the file inside a saml folder within the ZIP (saml/metadata.xml).

This bundle will allow all Elasticsearch containers to access the metadata file.
In the advanced configuration editor, update your Elasticsearch cluster configuration with the bundle you prepared in the previous step. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example:
```
{
...
  "resources": {
    "elasticsearch": [
      ...
        "plan": {
          ...
          "elasticsearch": {
            ...
            "user_bundles": [
            {
              "name": "saml-metadata",
              "url": "<MY_HOST_URL>/saml-metadata.zip",
              "elasticsearch_version": "*"
            }
          ]
        }
        ...
		
```
1. The URL for the bundle ZIP file must be always available. Make sure you host the plugin artifacts internally in a highly available environment.
These file locations are needed in the next step.

In this example, the SAML metadata file is located in the path /app/config/saml/metadata.xml:
```
$ tree .
.
└── saml
      └── metadata.xml
		
```
Adjust your saml realm configuration accordingly through Edit stack user settings:
```
idp.metadata.path: /app/config/saml/metadata.xml
		
```
1. The path to the SAML metadata file that was uploaded
Refer to SAML authentication for more details on SAML authentication.

Example: Add a custom JVM trust store bundle

If you are using SSL certificates signed by non-public certificate authorities, Elasticsearch is not able to communicate with the services using those certificates unless you import a custom JVM trust store containing the certificates of your signing authority into your Elastic Cloud Enterprise installation. You’ll need the trust store to access snapshot repositories like MinIO, for your Elastic Cloud Enterprise proxy, or to reindex from remote.

To import a JVM trust store:

Prepare the custom JVM trust store:
1. Pull the certificate from the service you want to make accessible:
```
openssl s_client -connect <server using the certificate> -showcerts
		
```
  1. The server address (name and port number) of the service that you want Elasticsearch to be able to access. This command prints the entire certificate chain to stdout. You can choose a certificate at any level to be added to the trust store.
2. Save it to a file with as a PEM extension.
3. Locate your JRE’s default trust store, and copy it to the current directory:
```
cp <default trust store location> cacerts
		
```
  1. Default JVM trust store is typically located in $JAVA_HOME/jre/libs/security/cacerts
  Tip
  
  Default trust store contains certificates of many well known root authorities that are trusted by default. If you only want to include a limited list of CAs to trust, skip this step, and simply import specific certificates you want to trust into an empty store as shown next
4. Use keytool command from your JRE to import certificate(s) into the keystore:
```
$JAVA_HOME/bin/keytool -keystore cacerts -storepass changeit -noprompt -importcert -file <certificate>.pem -alias <some alias>
		
```
  1. The file where you saved the certificate to import, and an alias you assign to it, that is descriptive of the origin of the certificate
  Important
  
  We recommend that you keep file name and password for the trust store as JVM defaults (cacerts and changeit respectively). If you need to use different values, you need to add extra configuration, as detailed later in this document, in addition to adding the bundle.
  
  You can have multiple certificates to the trust store, repeating the same command. There is only one JVM trust store per cluster currently supported. You cannot, for example, add multiple bundles with different JVM trust stores to the same cluster, they will not get merged. Add all certificates to be trusted to the same trust store
Create the bundle:
```
zip cacerts.zip cacerts
		
```
1. The name of the zip archive is not significant
Tip

A bundle may contain other contents beyond the trust store if you prefer, but we recommend creating separate bundles for different purposes.
In the advanced configuration editor, update your Elasticsearch cluster configuration with the bundle you prepared in the previous step. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example:
```
{
...
  "resources": {
    "elasticsearch": [
     ...
        "plan": {
         ...
          "elasticsearch": {
            ...
            "user_bundles": [
            {
              "name": "custom-ca-certs",
              "url": "<MY_HOST_URL>/cacerts.zip",
              "elasticsearch_version": "*"
            }
          ]
        }
    ...
		
```
1. The URL for the bundle ZIP file must be always available. Make sure you host the plugin artefacts internally in a highly available environment.
2. Wildcards are allowed here, since the certificates are independent from the Elasticsearch version.
(Optional) If you prefer to use a different file name and/or password for the trust store, you also need to add an additional configuration section to the cluster metadata before adding the bundle. This configuration should be added to the Elasticsearch cluster data section of the advanced configuration page:
```
"jvm_trust_store": {
  "name": "<filename included into bundle>",
  "password": "<password used to create keystore>"
}
		
```
1. The name of the trust store must match the filename included into the archive
2. Password used to create the trust store
Important
- Use only alphanumeric characters, dashes, and underscores in both file name and password.
- You do not need to do this step if you are using default filename and password (cacerts and changeit respectively) in your bundle.

Example: Add a custom GeoIP database bundle

Prepare a ZIP file with a custom bundle that contains a: GeoLite2 database. The folder has to be named ingest-geoip, and the file name can be anything that is appended -(City|Country|ASN) with the mmdb file extension, and it must have a different name than the original name GeoLite2-City.mmdb.

The file my-geoip-file.zip should look like this:
```
$ tree .
.
└── ingest-geoip
    └── MyGeoLite2-City.mmdb
		
```
Copy the ZIP file to a webserver that is reachable from any allocator in your environment.

In the advanced configuration editor, update your Elasticsearch cluster configuration with the bundle you prepared in the previous step. Modify the user_bundles JSON attribute of each Elasticsearch instance type as shown in the following example.

		{
...
  "resources": {
    "elasticsearch": [
     ...
        "plan": {
         ...
          "elasticsearch": {
            ...
            "user_bundles": [
            {
              "name": "custom-geoip-db",
              "url": "<MY_HOST_URL>/my-geoip-file.zip",
              "elasticsearch_version": "*"
            }
          ]
        }
		
	

To use this bundle, you can refer it in the GeoIP processor of an ingest pipeline as MyGeoLite2-City.mmdb under database_file such as:
```
...
{
  "geoip": {
    "field": ...
    "database_file": "MyGeoLite2-City.mmdb",
    ...
  }
}
...
		
```

Example: Add a custom synonyms bundle

Prepare a ZIP file with a custom bundle that contains a dictionary of synonyms in a text file.

The file synonyms.zip should look like this:
```
$ tree .
.
└── dictionaries
    └── synonyms.txt
		
```
Copy the ZIP file to a webserver that is reachable from any allocator in your environment.

		{
...
  "resources": {
    "elasticsearch": [
     ...
        "plan": {
         ...
          "elasticsearch": {
            ...
            "user_bundles": [
            {
              "name": "custom-synonyms",
              "url": "<MY_HOST_URL>/synonyms.zip",
              "elasticsearch_version": "*"
            }
          ]
        }
		
	