Skip to main content

The Forrester Wave™: XDR Platforms, Q2 2026

Access report
About usPartnersSupport|Login
Elastic logo
Start free trialContact sales

What's new in ECK 3.5

Dynamic namespaces, pause orchestration, and mutual TLS everywhere

By
Kostas Stamatakis
Summary
  • Dynamic namespaces: Manage namespaces with a label selector instead of a static list, onboarding and offboarding namespaces live with no operator restart.
  • Pause orchestration: Temporarily suspend spec-driven changes during maintenance windows while keeping essential housekeeping running.
  • Mutual TLS across all stack components: Every component that connects to Elasticsearch can now present an ECK-managed client certificate automatically.
  • StackConfigPolicy improvements: Define Elasticsearch roles declaratively and create reusable policies with dynamic substitution variables.

Elastic Cloud on Kubernetes (ECK) 3.5 makes the Elasticsearch Kubernetes operator more flexible to run, safer to maintain, and more secure by default. The headline feature, one of many in this release, is namespace management: a single kubectl label. Label a namespace, and the operator starts managing it; remove the label, and it stops with no operator restart and no configuration change. Let's take a closer look.

The challenge: Operating Elastic at scale on Kubernetes

Platform teams running Elastic on Kubernetes rarely operate a single, static environment. Namespaces come and go as teams onboard, maintenance windows require careful coordination between infrastructure changes and the operator's reconciliation loop, and security requirements increasingly demand mutual TLS between every component. ECK 3.5 removes the operator restarts, configuration file changes, and rolling restarts of Elasticsearch that these workflows previously involved.

Dynamic namespaces: Label-selector-based namespace management

ECK 3.5 changes namespace handling. The operator can now decide which namespaces it manages based on their labels.

Historically, ECK gave you two options: manage every namespace in the cluster or manage a static list of namespaces baked into the operator configuration. The static list works well for stable environments. In platform-as-a-service scenarios, where teams and their namespaces are onboarded and offboarded continuously, each change to the list means updating the operator configuration and restarting the operator.

ECK 3.5 introduces dynamic namespace handling. So, instead of a list of names, you configure a standard Kubernetes label selector, and the operator evaluates it against namespace labels at runtime.

With the Helm chart, enabling it is a single value:

managedNamespaceSelector:
  matchLabels:
    eck-managed: "true"

matchExpressions are supported too, so you can scope the operator to entire classes of environments:

managedNamespaceSelector:
  matchExpressions:
    - key: environment
      operator: In
      values: [production, staging]

The selector replaces the static managedNamespaces list and requires the operator to run with cluster-wide permissions since it must watch namespaces across the whole cluster.

From that point on, namespace management becomes a labeling operation:

# Onboard: the operator immediately picks up Elastic resources in the namespace
kubectl label namespace my-namespace eck-managed=true

# Offboard: the operator stops reconciling resources in the namespace
kubectl label namespace my-namespace eck-managed-

When a namespace gains matching labels, it is onboarded immediately. The operator enumerates the Elastic resources in it and starts reconciling them. When its labels stop matching, it is offboarded, so the operator stops managing its resources. Both transitions happen live. The operator watches Namespace objects and reacts to label changes as they occur with no operator restart required.

A few design decisions are worth calling out:

  • Filtering is enforced in depth: Events from non-matching namespaces are dropped before they reach a controller, reconciliation requests for namespaces that have left the scope are skipped, and the operator's Kubernetes client filters reads so that out-of-scope resources are invisible even to code paths that bypass the event-driven filtering.

  • Offboarding is non-destructive: Offboarding a namespace does not delete or modify the Elastic resources in it. Your Elasticsearch clusters and Kibana instances keep running; the operator simply stops operating them until the namespace is onboarded again. Bear in mind that operator-driven housekeeping, such as certificate rotation, stops with it.

  • The operator's own namespace is always in scope: It remains included regardless of its labels, so cluster-wide concerns like license management can never be accidentally offboarded.

Dynamic namespace handling is an Enterprise feature. Visit the dynamic namespace handling documentation for the full configuration reference, including YAML-manifest installation and license handling details.

Pause orchestration: Safe maintenance windows

ECK 3.5 introduces a purpose-built annotation that pauses spec-driven orchestration during maintenance windows while keeping essential housekeeping running:

kubectl annotate elasticsearch quickstart eck.k8s.elastic.co/pause-orchestration=true --overwrite

Maintenance operations, such as draining Kubernetes nodes, applying infrastructure changes, or migrating storage, have always sat uneasily next to an operator that continuously reconciles the world back to its desired state. ECK previously offered the eck.k8s.elastic.co/managed: "false" annotation for this, which pauses reconciliation entirely, including background work like certificate rotation and secret management.

Pausing orchestration is more selective:

PausedContinues
Scaling up and downCertificate rotation
Rolling upgradesService reconciliation
Configuration rolloutsUser and secret management
Volume expansionHealth monitoring and status updates

 

The resource's status reports an OrchestrationPaused condition, so the state is visible at a glance, and the annotation value is validated by the webhook to catch typos before they silently do nothing.

When the maintenance window closes, remove the annotation and any pending spec changes are applied immediately. The annotation is supported on all ECK-managed resource types, and the old managed: "false" annotation is now deprecated in its favor.

Visit the pause orchestration documentation for more details.

Mutual TLS expansion across all stack components

ECK 3.4 introduced support for configuring Elasticsearch to require client certificates with Kibana as the first component able to present one. ECK 3.5 completes the picture; every stack component that connects to Elasticsearch now automatically receives an ECK-managed client certificate and presents it when connecting, which covers: 

  • APM Server 

  • Beats

  • Enterprise Search 

  • Elastic Maps Server

  • Logstash

  • Standalone Elastic Agent

  • Fleet Server

  • Stack monitoring sidecars

  • AutoOps agent 

As in 3.4, Elasticsearch client certificate authentication requires an Enterprise license.

For fleet-managed agents, Fleet Server propagates the client certificate information to all connected agents automatically with no additional configuration required.

Going one step further, Fleet Server itself can now be configured to require client certificates from the Elastic Agents connecting to it, extending mutual TLS to the agent-to-Fleet-Server hop as well. This is an Enterprise feature and requires a recent Fleet Server version (i.e., 8.19.19, 9.3.8, 9.4.4, or 9.5.0 and later).

Visit the Elasticsearch client certificate authentication documentation and the Fleet Server client certificate authentication documentation for more details.

StackConfigPolicy: Declarative roles and reusable policies

StackConfigPolicy, ECK's mechanism for applying consistent configuration across fleets of Elasticsearch clusters and Kibana instances, gains two capabilities in this release.

Elasticsearch role definitions

The new securityRoles field lets you define custom Elasticsearch roles declaratively within a policy and apply them consistently across all targeted clusters:

apiVersion: stackconfigpolicy.k8s.elastic.co/v1alpha1
kind: StackConfigPolicy
metadata:
  name: shared-roles
spec:
  resourceSelector:
    matchLabels:
      env: production
  elasticsearch:
    securityRoles:
      click_admins:
        indices:
          - names: ["events-*"]
            privileges: ["read"]

ECK merges the definitions into the roles.yml file mounted on each Elasticsearch pod, and Elasticsearch hot-reloads them with no pod restart required.

Dynamic substitution variables

The new variablesFrom field lets you load key-value pairs from ConfigMaps and Secrets and reference them as ${VAR} or ${VAR:-default} expressions in the policy's elasticsearch and kibana fields. A single policy definition can now be reused across environments with different values, and ECK watches all referenced sources and reconciles automatically when they change.

Visit the Stack Configuration Policy documentation for more details.

Simplified container resources: Set CPU and memory without the podTemplate boilerplate

Setting CPU and memory on an Elastic workload is one of the most common customizations ECK users make, and it used to require navigating a four-level nested path: podTemplate.spec.containers[name=<main>].resources. That is a lot of YAML for a memory request.

ECK 3.5 adds a top-level resources shorthand to every pod-owning ECK CRD. Compare the two ways of giving an Elasticsearch NodeSet 8Gi and 4 CPUs:

# Before: four levels of nesting
spec:
  nodeSets:
  - name: default
    count: 3
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          resources:
            requests: { memory: 8Gi, cpu: 4 }
            limits:   { memory: 8Gi, cpu: 4 }

# After: a top-level field
spec:
  nodeSets:
  - name: default
    count: 3
    resources:
      requests: { memory: 8Gi, cpu: 4 }
      limits:   { memory: 8Gi, cpu: 4 }

Both produce exactly the same container resources. For Elasticsearch the field lives on each NodeSet (spec.nodeSets[].resources), so heterogeneous topologies keep per-tier sizing. For all other workloads (e.g., Kibana, APM Server, Beats, Elastic Agent, Enterprise Search, Logstash, Elastic Maps Server, and more) it sits directly under spec:

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: my-kibana
spec:
  version: 9.5.0
  count: 1
  elasticsearchRef:
    name: my-cluster
  resources:
    requests: { memory: 1Gi, cpu: 500m }

The shorthand composes cleanly with existing manifests; it targets only the main application container. And if both the shorthand and podTemplate set CPU or memory, the shorthand wins while non-CPU/memory keys, such as ephemeral-storage, are preserved. The admission webhook emits a non-blocking warning when the two overlap, so accidental double definitions don't go unnoticed. Existing manifests are unaffected: the new field is entirely opt-in, and the Elasticsearch autoscaler is fully aware of it.

Reduced operator memory footprint

Large, busy Kubernetes clusters put pressure on any operator's cache. ECK 3.5 ships two complementary improvements:

  • The controller-runtime cache is now automatically scoped to only watch core workload resources like Pods, StatefulSets, Deployments, DaemonSets, PodDisruptionBudgets that carry the ECK type label, avoiding the cost of caching unrelated workloads running in the same cluster.

  • A new opt-in flag, --restrict-watched-resources, further narrows the cache for secrets, services, and configmaps to those explicitly labeled with eck.k8s.elastic.co/watched=true, which reduces memory usage and API server load in clusters with large numbers of user-managed resources of those types.

Together with the managed-fields stripping introduced in 3.4, these changes make the operator's footprint predictable even in clusters where Elastic workloads share space with thousands of unrelated resources.

Other notable improvements in ECK 3.5

  • Hot-reload of secure settings: For Elasticsearch 9.5 and later, the opt-in eck.k8s.elastic.co/file-based-secure-settings: "true" annotation delivers spec.secureSettings through Elasticsearch file-based settings, so updating a credential no longer triggers a rolling restart. This is intended for reloadable settings, such as snapshot repository credentials.

  • AutoOps agent collector configuration: The AutoOpsAgentPolicy resource now exposes spec.config and spec.configRef fields for tuning which metricsets the AutoOps agent collects and at what interval directly from the CRD. See the AutoOps data collection documentation.

  • Kibana Spaces support within Fleet integration policies: This enables space-aware Fleet setups managed through ECK.

  • Cert-manager compatibility: Custom CA secret parsing has been relaxed to accept secrets in the format produced by cert-manager.

  • Map support for extraObjects: Helm charts now support maps, allowing additional manifests to be composed and overridden across values files.

  • FIPS: Now that Go's native FIPS module is FIPS 140-3 certified, the FIPS-enabled operator image builds with Go's native FIPS 140-3 mode instead of the BoringCrypto-based toolchain used in previous releases. The result is a fully static binary with the dynamically linked image variant no longer needed.

  • Kibana readiness probes: They now use the Kibana status API for more accurate health signals.

Getting started with ECK 3.5

Ready to upgrade? ECK 3.5.0 is available now. Existing users can follow the upgrade documentation; the operator upgrade preserves your running Elastic Stack workloads.

New to ECK? The quickstart guide gets you from zero to a running Elasticsearch cluster in minutes. Features marked as Enterprise, such as dynamic namespaces, can be evaluated with a free Enterprise trial license.

For the complete list of features, enhancements, and fixes in this release, check the ECK release notes.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

Sign up for Elastic Cloud free trial

Spin up a fully loaded deployment on the cloud provider you choose. As the company behind Elasticsearch, we bring our features and support to your Elastic clusters in the cloud.

Start free trial