Elastic Common Schema (ECS) provides a consistent way to structure your data in Elasticsearch, facilitating the analysis of data from diverse sources. With ECS, analytics content such as dashboards and detection rules can be applied more broadly, searches can be crafted more narrowly, and field names are easier to remember.
Whether performing interactive analysis (e.g., search, drill-down and pivoting, visualization) or automated analysis (e.g., alerting, detection rules and machine learning-driven anomaly detection), you need to be able to uniformly examine your data. But unless your data originates from only one source, you face formatting inconsistencies due to disparate data types and heterogeneous environments with diverse vendor standards.
ECS is an open source, community-driven specification that defines a common set of fields, their Elasticsearch data types, allowed values and usage hierarchy for data ingested into Elasticsearch. It unifies all modes of analysis available in Elastic, including search, drill-down and pivoting, data visualization, machine learning-based anomaly detection, detection rules and alerting
ECS reduces the amount of time you spend on developing analytics content. Instead of creating new searches and dashboards each time your organization adds a new data source, you’ll be able to continue leveraging your existing searches and dashboards. ECS will also make it far easier for your environment to adopt analytics content directly from other parties that use ECS, whether Elastic, a partner, or an open source project,
Elastic provides out-of-the-box integrations to stream in logs, events, metrics, traces, content, and more from your apps, endpoints, infrastructure, cloud, network, workplace tools, and every other common source in your ecosystem. These integrations ensure that you can interact with your data within Elastic solutions such as Security and Observability, amongst other areas of the Elastic stack.
Data ingested from these integrations are already mapped to ECS. You simply enable the integration, ingest data and you can begin to interact with your ECS-formatted data.
While Elastic integrations automatically map data to ECS, you likely have other data sources that you would like to normalize to ECS, to reap the benefits there as well. There are lots of options available to help you map your data to ECS. This blog post provides a great example of mapping security data source to ECS.
ECS is an evolving schema with regular updates to address new use cases, based on community feedback.