Trust a private certificate authority for external inference endpoints

If your model provider presents a TLS certificate signed by a private or internal certificate authority (CA), Elasticsearch rejects the connection. Because Elasticsearch sends a test request when you create an endpoint, the error appears as soon as you try to save it:

		PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
		
	

Elasticsearch sends requests to external inference services through the default JVM trust store. Unlike Kibana connectors, inference endpoints have no per-endpoint SSL settings, and you can't turn off certificate verification. To fix the error, add your CA certificate to the trust store that the Elasticsearch JVM uses.

Note

The xpack.inference.elastic.http.ssl.* settings apply only to the Elastic Inference Service (EIS). They have no effect on external inference services.

Before you begin

Have the CA certificate that signed the TLS certificate of your model provider available in PEM format.

Keep the following in mind before you change the trust store:

The trust store applies to the whole node. Every outbound TLS connection that uses the default JVM trust store trusts your CA, not only inference endpoints.
Add the CA on every node in the cluster. Any node can send inference requests, so a missing CA on one node causes intermittent failures.
The JVM reads the trust store at startup, so you need to restart each node after you change it.
Trusting the CA doesn't relax hostname verification. The server certificate must still include a subject alternative name (SAN) that matches the host in the endpoint url.

Add your CA to the Elasticsearch trust store

Create a dedicated trust store that contains the default public CAs plus your own CA, then point the JVM at it.

Using a separate trust store, rather than importing your CA into the bundled cacerts file directly, means your change survives upgrades. Each Elasticsearch upgrade ships a new JDK that replaces the bundled cacerts file.

Warning

The custom trust store replaces the default JVM trust store. Start from a copy of the bundled JDK's cacerts file, as shown in the following steps. A trust store that contains only your CA breaks every other TLS connection the node makes, such as connections to snapshot repositories and EIS.

Copy the bundled JDK trust store into a new PKCS#12 file in your Elasticsearch configuration directory:

		$ES_HOME/jdk/bin/keytool -importkeystore \
  -srckeystore $ES_HOME/jdk/lib/security/cacerts \
  -srcstorepass changeit \
  -destkeystore $ES_PATH_CONF/inference-truststore.p12 \
  -deststoretype PKCS12 \
  -deststorepass <TRUSTSTORE_PASSWORD>
		
	

Import your CA certificate into the new trust store:

		$ES_HOME/jdk/bin/keytool -importcert -noprompt \
  -alias my-internal-ca \
  -file /path/to/ca.crt \
  -keystore $ES_PATH_CONF/inference-truststore.p12 \
  -storepass <TRUSTSTORE_PASSWORD>
		
	

Create a file named inference-truststore.options in the jvm.options.d directory of your Elasticsearch configuration directory, and use absolute paths:
```
-Djavax.net.ssl.trustStore=/path/to/config/inference-truststore.p12
-Djavax.net.ssl.trustStorePassword=<TRUSTSTORE_PASSWORD>
-Djavax.net.ssl.trustStoreType=PKCS12
		
```
For more information, refer to JVM settings.
Make sure the elasticsearch user can read the trust store file.
Repeat these steps on every node, then restart each node.

Tip

If you run Elasticsearch in Docker, you can instead build a custom image that adds your CA to the operating system trust anchors. In the official images, the JDK cacerts file links to the operating system trust bundle, so the JVM picks up the change:

		FROM docker.elastic.co/elasticsearch/elasticsearch:9.5.5
USER root
COPY ca.crt /etc/pki/ca-trust/source/anchors/my-internal-ca.crt
RUN update-ca-trust extract
USER elasticsearch
		
	

For -wolfi images, copy the certificate to /usr/local/share/ca-certificates/ and run update-ca-certificates instead of update-ca-trust extract.

Upload a custom JVM trust store as a bundle and add it to your deployment. For the steps, refer to Add a custom JVM trust store bundle.

Store a custom JVM trust store in a Kubernetes Secret, mount it into the Elasticsearch Pods, and configure the JVM to use it with the ES_JAVA_OPTS environment variable.

The closest example is Use S3-compatible services, which sets up a custom trust store for a snapshot repository. To adapt it for inference endpoints:

Follow steps 1 to 4 to build the trust store and store it in a Secret. In step 2, use the CA certificate of your model provider.
In step 6, add only the trust store volume, the volume mount, and the ES_JAVA_OPTS environment variable. Leave out secureSettings.
Skip step 5 and step 7, which set up the snapshot repository.