Upgrade ECKedit
This page provides instructions on how to upgrade ECK.
For Elastic Stack upgrade, check Upgrade the Elastic Stack version.
Upgrade to ECK 1.9.1edit
ECK reached general availability (GA) status with the release of version 1.0.0. The latest available GA version is 1.9.1. It is compatible with the previous GA releases (1.0.x and higher) and the beta release (1.0.0-beta1), and can be upgraded in-place (with a few exceptions) by applying the new set of deployment manifests. Previous alpha releases, up to and including version 0.9.0, are not compatible with the GA and beta releases and require extra work to upgrade.
Before upgrading, refer to the release notes to make sure that the release does not contain any breaking changes that could affect you. The release highlights document provides more details and possible workarounds for any breaking changes or known issues in each release.
Note that the release notes and highlights only list the changes since the last release. If you are skipping over any intermediate versions during the upgrade — such as going directly from 1.0.0-beta1 to 1.9.1 — review the release notes and highlights of each of the skipped releases to fully understand all the breaking changes you might encounter during and after the upgrade.
Upgrade from beta or previous GA releases to ECK 1.9.1edit
The upgrade process results in an update to all the existing managed resources. This triggers a rolling restart of all Elasticsearch and Kibana pods. If you have a large Elasticsearch cluster or multiple Elastic Stack deployments, the rolling restart could cause a performance degradation. When you plan to upgrade ECK for production workloads, take into consideration the time required to upgrade the ECK operator plus the time required to roll all managed workloads and Elasticsearch clusters. Furthermore, Guidance is available on controlling this process more gracefully.
If you are running Openshift 3.11 or a Kubernetes version older than 1.13, and want to upgrade from beta, jump to the upgrade instructions for older Kubernetes releases.
Release 1.7.0 moves the CustomResourceDefinitions (CRD) used by ECK to the v1 version. If you upgrade from a previous version of ECK, the new version of the CRDs replaces the existing CRDs. If you cannot remove the current ECK installation because you have production workloads that must not be deleted, the following approach is recommended.
If you are installing using the YAML manifests: replace existing CRDs.
kubectl replace -f https://download.elastic.co/downloads/eck/1.9.1/crds.yaml
If you skipped a release in which new CRDs where introduced, you will see an error message similar to Error from server (NotFound): error when replacing "config/crds.yaml": customresourcedefinitions.apiextensions.k8s.io ... not found. To add the missing CRDs run
kubectl create -f https://download.elastic.co/downloads/eck/1.9.1/crds.yaml`
Then upgrade the remaining objects with the operator manifest:
kubectl apply -f https://download.elastic.co/downloads/eck/1.9.1/operator.yaml
If you are running a version of Kubernetes before 1.16: use the legacy manifests.
kubectl replace -f https://download.elastic.co/downloads/eck/1.9.1/crds-legacy.yaml kubectl apply -f https://download.elastic.co/downloads/eck/1.9.1/operator-legacy.yaml
If you are using Helm: force upgrade the CRD chart to move to the v1 CRDs.
helm upgrade elastic-operator elastic/eck-operator-crds -n elastic-system --force
Then upgrade the main chart as usual:
helm upgrade elastic-operator elastic/eck-operator -n elastic-system
If you are using ECK through an OLM-managed distribution channel like operatorhub.io or the OpenShift OperatorHub then the CRD version upgrade will be handled by OLM for you and you do not need to take special action.
This will update the ECK installation to the latest binary and update the CRDs and other ECK resources in the cluster. If you are upgrading from the beta version, ensure that your Elasticsearch, Kibana, and APM Server manifests are updated to use the v1 API version instead of v1beta1 after the upgrade.
Control rolling restarts during the upgradeedit
Upgrading the operator results in a one-time update to existing managed resources in the cluster. This triggers a rolling restart of pods by Kubernetes to apply those changes. If you have a very large Elasticsearch cluster or multiple Elastic Stack deployments, this rolling restart might be disruptive or inconvenient. To have more control over when the pods belonging to a particular deployment should be restarted, you can add an annotation to the corresponding Elasticsearch, Kibana, or APM Server resources to temporarily exclude them from being managed by the operator. When the time is convenient, you can remove the annotation and let the rolling restart go through.
Once a resource is excluded from being managed by ECK, you will not be able to add/remove nodes, upgrade Stack version, or perform other orchestration tasks by updating the resource manifest. You must remember to remove the exclusion to ensure that your Elastic Stack deployment is continually monitored and managed by the operator.
Exclude Elastic resources from being managed by the operator.
ANNOTATION='eck.k8s.elastic.co/managed=false' # Exclude a single Elasticsearch resource named "quickstart" kubectl annotate --overwrite elasticsearch quickstart $ANNOTATION # Exclude all resources in the current namespace kubectl annotate --overwrite elastic --all $ANNOTATION # Exclude all resources in all of the namespaces: for NS in $(kubectl get ns -o=custom-columns='NAME:.metadata.name' --no-headers); do kubectl annotate --overwrite elastic --all $ANNOTATION -n $NS; done
Once the operator has been upgraded and you are ready to let the resource become managed again (triggering a rolling restart of pods in the process), remove the annotation.
Resume Elastic resource management by the operator.
RM_ANNOTATION='eck.k8s.elastic.co/managed-' # Resume management of a single Elasticsearch cluster named "quickstart" kubectl annotate elasticsearch quickstart $RM_ANNOTATION
The ECK source repository contains a shell script to assist with mass addition/deletion of annotations.
Upgrade from beta to GA on older versions of Kubernetesedit
If you are running Openshift 3.11 or a Kubernetes version older than 1.13 and have existing resources created using the v1beta1 API version that are being updated to v1 API version as part of the ECK upgrade, then in-place upgrade is not possible due to an upstream bug. You have to completely uninstall the existing ECK operator (which will cause all existing Elastic Stack applications to be deleted) and install the new version of the operator. Make sure to backup your existing manifests and take snapshots of Elasticsearch clusters before uninstalling ECK. Then you can install the new version of ECK, update your manifests to the v1 API version, re-create the Elastic Stack deployments, and restore data from the snapshots.