The mystery stress your heap chart can't see: AutoOps now watches vector off-heap memory
Dense vectors use off-heap memory your heap chart never shows. AutoOps detects memory pressure before vector RAM stress causes OOM.
AutoOps now raises a Vector memory pressure insight when dense vector off-heap footprint, heap heat, and operational stress converge on the same Elasticsearch node. We validated on a 4 GiB node under sustained k-nearest neighbor (kNN) ingest: The insight fired at ~75% heap with thread-pool stress, roughly an hour before saturation. Heap charts alone still looked moderate at that point. Dense vectors for kNN live outside the Java heap, so heap monitoring and circuit breakers never show the full vector RAM picture. Below, we walk through what the insight measures and why heap on its own misses this. We also discuss what to do when it fires.
Why dense vectors create off-heap memory pressure that heap charts miss
Semantic search and kNN rely on dense_vector fields. Elasticsearch stores much of that data in off-heap memory. It’s related to how the Java Virtual Machine (JVM) operates, but it isn’t the same thing as heap usage.
In production, the heap versus off-heap split shows up in a familiar pattern:
Heap looks fine for weeks, while the dense vector off-heap footprint quietly grows.
Heap circuit breakers stay quiet or only spike late because the pressure sits outside the JVM.
kNN search and bulk ingest slow down, queues build, and nothing on the dashboard points at vector RAM as the cause.
Heap limits protect Java allocations. They don’t tell you whether vector off-heap footprint still fits the RAM envelope that your deployment actually runs in. AutoOps already watches cluster health broadly; Vector memory pressure adds a focused read for vector-heavy nodes when memory and load signals line up.
How AutoOps measures vector RAM, heap, and headroom
AutoOps works from the same node stats metrics you already use for Stack Monitoring. For each node, it tracks three derived numbers:
Symbol | Meaning | Source (typical) | Chart (see below) |
|---|---|---|---|
V | Vector off-heap footprint |
| First, green line |
A | Available RAM in the product view | Delta between | Second, green line |
H | Headroom | A − V (headroom_bytes) | First, blue line |
H > 0 means there’s a modeled runway: Vector use still fits comfortably in that accounting. H ≤ 0 means that you’re in a compression regime: Vector footprint (V) meets or exceeds the free RAM (A) picture that AutoOps can align in telemetry. On small tiers, that can be common under load. The insight emphasizes trends, growth in vector off-heap footprint, and corroborating stress, not a single negative snapshot.
AutoOps also tracks a compression regime flag (fraction of recent samples where H ≤ 0), so brief flickers don’t dominate the story (see third chart below):
How vector memory pressure detection works: Expansion and compression
Vector memory pressure is a single HIGH severity AutoOps event, which adapts to the compression regime:
Expansion (H > 0): Emphasize shrinking headroom, hence a growing vector off-heap footprint.
Compression (H ≤ 0): Emphasize ΔV, heap context, I/O, and latency. "Hours until H hits zero" isn’t the main narrative when headroom is already gone.
The detector requires three layers before it fires:
Memory carriers: Compression regime, shrinking headroom, or sustained growth in vector off-heap footprint.
Operational latch: Search or indexing latency versus rolling baselines, filesystem read stress (paired with latency or heap), indexing throttle, thread-pool queues or rejections, segment creep, or heap circuit breakers when paired with other stress, as circuit breakers alone don’t provide enough evidence to be escalated without corroborating stress.
Heap hot: Heap usage elevated versus a 24-hour rolling median on that node, so compression alone on a calm heap doesn’t fire the insight.
That pairing is intentional. Vector pressure without load might be capacity planning, and load without vector pressure might be a different root cause. Together, vector memory, operational stress, and heap heat surface the vector RAM story when the node is actually in trouble, not on every compressed mapping while the heap stays normal.
Validation: Memory pressure detection on a 4 GiB node under kNN load
We stress-tested vector memory pressure detection on 4 GiB Elastic Cloud Hosted deployments with throttled dense-vector ingest (~2,000 docs per minute) and steady kNN search (~8 queries per second). Across Hierarchical Navigable Small World (HNSW), Better Binary Quantization (BBQ) HNSW, and DiskBBQ mapping profiles over 24–48 hours:
Vector off-heap footprint grew from near zero to about 6 GiB on the tightest runs (more than 4.7 million vectors indexed) in both HNSW test runs.
Nodes spent most of each run in compression (H ≤ 0), which is expected when vector footprint exceeds total RAM in this model.
Vector memory pressure stayed off while heap held near 50%, even with compression and pool stress building.
On both HNSW and BBQ HNSW, the insight fired once heap climbed past ~75% with memory compression and thread-pool queue stress, roughly an hour before heap neared saturation. Node out of memory (OOM) and circuit breakers followed in the same window, as did slow search/indexing. As we can see on the dashboard below, performance drops drastically due to corroborating stress toward the end of the test run:
- On DiskBBQ, where compression was chronic but heap stayed normal, the insight didn’t fire, since storage rather than memory was the limiting factor. Disk and watermark signals are the right path to follow for that profile. As we can see on the screenshot below, all indicators stayed steady with constant performance throughout the test, even though we filled up the disk with more than 70 million vectors on the same instance type:
That timing is the point: Operators get a vector-first read tied to real RAM stress, with subsystem context, rather than an alert on every compressed index or only a red heap chart after the node is fighting on every front.
What to do when AutoOps raises vector memory pressure
Here’s the insight that AutoOps now raises when it detects vector memory pressure:
Recommendations in the product map to concrete actions:
Reduce vector footprint where quality allows: Fewer dimensions, quantized mappings, archive or split indices, reindex with a leaner mapping.
Tune kNN load: Lower
num_candidates, reduce concurrent query rate, narrow filtered kNN where possible.Consider DiskBBQ when HNSW in RAM is the bottleneck (evaluate recall/latency trade-offs for your use case). If you’re already on DiskBBQ and the heap memory is calm, treat disk and watermark insights as the primary signals. Note that DiskBBQ requires an Enterprise license.
Right-size RAM when vector off-heap footprint (V) trends up and headroom stays tight.
AutoOps links affected nodes and summarizes regime and stress in plain language. Treat it as “act now, rather than waiting for red on every chart.”
Where AutoOps vector memory pressure monitoring is available
Vector memory pressure is available wherever AutoOps runs against Elasticsearch 9.2+, including:
Elastic Cloud Hosted (ECH).
Elastic Cloud Serverless (coming soon).
Self-managed via Cloud Connect.
AutoOps is included at all subscription levels for supported deployment types and doesn’t consume ECUs on ECH.
Frequently Asked Questions
Why does AutoOps report vector memory pressure when my heap chart still looks moderate?
The insight fires when heap is hot relative to that node's own 24-hour baseline, not only when heap hits 99%. Dense vectors for kNN use off-heap memory, so the JVM heap chart and heap circuit breakers never show the full vector footprint. AutoOps compares the vector off-heap footprint to the available RAM, requires operational stress (latency, queues, throttle), and waits for heap heat so you get a vector-specific signal as RAM stress builds, not a calm-heap false alarm on compressed mappings alone.
What does negative headroom (H ≤ 0) mean on a small node?
It means vector footprint meets or exceeds the free RAM picture in AutoOps accounting. On small tiers, that can happen under sustained ingest. Compression alone doesn’t fire the insight. AutoOps focuses on whether the situation is worsening and heap is elevated versus baseline. It also focuses on whether search, indexing, or pools show strain.
How is this different from a circuit breaker trip?
Circuit breakers protect heap-related work. Vector memory pressure watches off-heap vector data and pairs memory tightness with corroborating telemetry (latency, queues, throttle, I/O) and heap heat. A breaker trip may appear late or alongside vector stress; this insight surfaces the vector RAM story in the same crisis window, often before OOM and pool failures cascade.
What workloads trigger it most often?
Heavy `dense_vector` ingest, large `num_candidates` kNN, or both on memory-constrained nodes trigger it most often. Validation used steady bulk ingest plus kNN search; production mixes vary. Larger nodes may show expansion-regime runway before compression dominates. DiskBBQ-heavy nodes with normal heap are less likely to see this insight fire.
What should I do first when the insight fires?
Pay attention to the regime and subsystem lines in the insight. If vector footprint is growing and pools or latency are hot, ease kNN or bulk load and plan footprint reduction or RAM right-sizing. If the heap is also elevated, address both vector and JVM capacity rather than assuming one chart tells the whole story.
