We’re thrilled to announce the technical preview of the frozen tier in 7.12, enabling you to completely decouple compute from storage and directly search data in object stores such as AWS S3, Microsoft Azure Storage, and Google Cloud Storage. The next major milestone in our data tier journey, the frozen tier significantly expands your data reach by storing massive amounts of data for the long haul at much lower cost while keeping it fully active and searchable.
We've long supported multiple data tiers for data lifecycle management — hot for high speed and warm for lower cost and performance. Both leverage local hardware for storing your primary data and redundant copies. More recently, we’ve introduced the cold tier, which allows you to store up to twice the data on the same amount of hardware over the warm tier by eliminating the need to store your redundant copies locally. Although the primary data is still local for optimal performance, indices in the cold tier are backed by searchable snapshots stored in your object store for redundancy.
The frozen tier takes it a big step further by removing the need to store any data locally at all. Instead, it uses searchable snapshots to directly search data stored in the object store without any need to rehydrate it first. A local cache stores recently queried data for optimal performance on repeat searches. Storage costs go down significantly as a result — up to 90% over the hot or warm tiers, and up to 80% over the cold tier. The fully automated lifecycle of your data is now complete: from hot to warm to cold and then to frozen, all while ensuring you have the access and search performance you need at the lowest storage cost possible.
You never outgrow good data
Whether it’s for observability, security, or enterprise search, your IT data can keep growing at an exponential rate. It’s commonplace for organizations to ingest and search many terabytes a day. This data is critical not only for day-to-day success but also for historical reference. Unlimited lookback for security investigations, drilling into years of APM data for trend identification, or the occasional discovery for regulatory compliance are all key use cases for keeping your data around and accessible for longer periods of time. Satisfying these use cases, however, can quickly become very expensive if you don’t have the right tools or technology to store the data while ensuring it’s easily searchable.