Last year, the US Navy’s Capstone Design Concept for Information Superiority provided digital modernization guidance for the Department of the Navy (DON). The document sets forth the overarching goal: to “securely move any information from anywhere to anywhere” in order to achieve information superiority. The design concept also highlights two outcomes: customer experience and operational resilience.
In this article, we’ll explore the goal of “securely moving any information from anywhere to anywhere” — and suggest a way to enhance this approach to facilitate easier implementation in air-gapped and Denied, Degraded, Intermittent, and Limited (DDIL) environments, ensure data sovereignty, and scale with increasing data volume. We recommend that the DON augment its goal with a Global Data Mesh approach that enables data to be accessed and searched from anywhere, yet remain in its original location (at the edge, in a data center, within a specific region for clearance purposes) without needing to duplicate or move that data. Besides being practical and scalable, this distributed approach to data access better positions the DON to outpace its desired outcomes of operational resilience and customer experience.
Distributed search: Access your data without moving it
While we understand the intent behind the goal of being able to securely move data wherever it needs to go, moving data can come with risk and complexity. When you duplicate data or move it to a central repository, like a data lake, so that you can then access it from anywhere, you run the risk of data getting lost, in the wrong hands, and paying high and often duplicative data storage and transfer costs. The time and bandwidth required to operate a centralized data store quickly becomes untenable when dealing with the large volumes of operational and mission data that the DON uses on a daily basis. Operating in a DDIL environment can also make it riskier and more difficult to centralize your data.
The Elasticsearch® platform allows data to remain in its original point of collection, yet enables you to search across all data to find the exact information you need for your mission. Elasticsearch’s capability to search across environments is called Cross-Cluster Search (CCS). Without the need to duplicate or move data, this distributed approach — also known as a global data mesh — can significantly save on the costs of data transfer and storage, as only the query and its matching results would need to traverse the network.
Operational resilience is crucial for Navy missions, especially in DDIL and air-gapped environments. When operating in high latency or disconnected environments, a global data mesh approach is a benefit. Distributed architectures split the load across multiple servers or systems, which reduces reliance on a single, centralized data store or data lake — while at the same time facilitating data access at speed and scale. Even if a server or data center experiences an issue, the others can continue to operate.
A data store built with geo-distributed disaster recovery logic can further boost operational resilience. Disaster recovery strategies ensure that operations can continue even in the event of significant system failures or disasters, usually by maintaining backup systems or data centers in geographically diverse locations. In the event of a significant system failure or disaster in one region, operations can be quickly shifted to another location, minimizing downtime and data loss.
The Cross-Cluster Replication (CCR) capability in the Elasticsearch platform can be a key facilitator of operational resilience. CCR allows you to replicate certain data, or identify what data would need to be replicated if needed — reducing replication of entire databases. This functionality is especially valuable when co-locating data to and from the edge.
CCR enables a variety of mission-critical use cases, such as data recovery and high availability — enabling teams to withstand an outage in a region or data center. It also allows you to colocate data to and from the edge, allowing data to be closer to the user or application server, which reduces latencies that could affect your mission. CCR can also be used to replicate data from a large number of smaller clusters back to a centralized reporting cluster. This is useful when it may not be efficient to query across a large network.
The sheer amount of data used and produced by the DON today poses a significant challenge when teams need to find and access specific information quickly. That challenge is compounded by the fact that data is most often stored in different formats (e.g., images vs. documents vs. maps) and environments (e.g., cloud, on-prem). Not only that, but there are also significant variations in where data is collected, from ship to store, as well as data storage environments ranging from the C4I community to the enterprise. This inevitably leads to information silos, lack of interoperability, and generally complex environments.
When all your data is accessible via the same query, it’s easier for your teams to find exactly what they need, when and where they need it. Not having to painstakingly dig through mountains of internal documents and spreadsheets and correlate the information from different sources and teams not only saves time, but it also allows professionals to focus their attention on more strategic mission-focused work. Using Elasticsearch as a unified platform to bring together disparate data sources and data sets also dramatically reduces data silos and makes it easier for teams to collaborate with one another, whether at sea or on land.
By leveraging Elasticsearch as a global data mesh that enables you to access mission-critical data wherever it resides, you also unlock the ability to utilize the Elasticsearch platform for additional use cases, such as application performance monitoring, threat hunting, real-time situational awareness, and generative AI.
Taking data-backed action
In an information battlefield where data is both a weapon and a strategic asset, it’s crucial that data of all types and in all locations is easily accessible, findable, and actionable, especially in air-gapped and DDIL environments. We at Elastic stand at the ready to continue to support and partner with the DON to provide a seamless customer experience and top-of-the-line operational resilience.
To talk to an expert directly, contact us.
The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.