Elastic Cloud Data Ingestion

When it comes to delivering your data into Elastic Cloud, a variety of options are available for data ingestion. We’ve collected a series of resources to make it easier. From video to documentation, below you can check out a list of the fastest ways to get started.

Learn more

Don't have Elastic Cloud? Start a 14 day free trial.

videoImage

Data ingestion methods

By now you've probably spun up a deployment and might be wondering what's next. You likely have data that you want to add, known as ingesting or indexing, to Elasticsearch, so let's explore some data ingestion methods.

Enterprise Search

Observability

Security

Logs
Records and events from application servers, your web farm, IoT devices, or triggered actions by users

Application traces
Specialized logging that records information about your program's execution

Metrics
Point-in-time measurements of your application or system for monitoring, alerting, and analyzing trends

Security information and event management data (SIEM)
Security data from cloud, apps, network devices, hosts, servers, and more to power detection and response

Endpoint detection and response data (EDR)
Host activity data to identify threats and accelerate response

Files and catalogs: Pre-built search solutions
Simple-to-use, curated search experience with tunable relevance controls

Files and catalogs: Custom search solution
Customized data store and search experience leveraging Elasticsearch APIs and clients as well as file upload via Kibana

Logs
Records and events from application servers, your web farm, IoT devices, or triggered actions by users

Application traces
Specialized logging that records information about your program's execution

Metrics
Point-in-time measurements of your application or system for monitoring, alerting, and analyzing trends

Security information and event management data (SIEM)
Security data from cloud, apps, network devices, hosts, servers, and more to power detection and response

Endpoint detection and response data (EDR)
Host activity data to identify threats and accelerate response

Files and catalogs: Pre-built search solutions
Simple-to-use, curated search experience with tunable relevance controls

Files and catalogs: Custom search solution
Customized data store and search experience leveraging Elasticsearch APIs and clients as well as file upload via Kibana

Wondering what the difference between Beats (i.e., Filebeat, Metricbeat, etc.) and Elastic Agent is? Check out our documentation.

Ship. Visualize. Analyze.

Data ingestion in Elastic Cloud is easy. Learn to ship data quickly so you can create robust visualizations for faster analysis.

Filebeat

Filebeat

Filebeat is a lightweight shipper for log ingestion that allows you to forward and centralize log data and other files. Installed as an agent on your hosts or servers, Filebeat monitors log files or locations that you specify, collects log events, and forwards them to Elasticsearch or Logstash for indexing.

Elastic Agent

Elastic Agent

Elastic Agent is a single, unified agent that simplifies the data ingestion process for logs, metrics, APM traces, synthetics, and securing your system. With Elastic Agent, you can quickly deploy across your ecosystem and centrally manage your agents through Kibana.

APM Agent

APM Agent

Elastic APM is an application performance monitoring system built on the Elastic Stack. With the APM Agent, you can easily collect and stream application performance metrics to your APM Server, where they are processed and stored in Elasticsearch. This allows you to easily monitor software services and applications in real time to pinpoint and fix performance problems quickly.

Metricbeat

Metricbeat

Metricbeat is a lightweight shipper that you can install on your servers to periodically collect metrics from the operating system and from services running on the server. Metricbeat takes the metrics and statistics that it collects and ships them to the output that you specify, such as Elasticsearch or Logstash.

Web crawler

Web crawler

Elastic App Search web crawler is an easy yet powerful way to ingest publicly available web content so it becomes instantly searchable on your website. The web crawler fetches content, extracts it, and transforms your web pages into search documents that are automatically indexed. Simply provide a URL and let the web crawler do the rest.

Content Source Connector: Workplace Search

Content Source Connector: Workplace Search

In Elastic Workplace Search, use prebuilt integrations to unify all content platforms across your organization into a single view, set up users and groups with access to the content most important to them, and tune relevance by prioritizing data sources by group.

Language Clients: Prebuilt search solutions

Language Clients: Prebuilt search solutions

Elastic Enterprise Search provides HTTP APIs. Developers use these APIs to integrate Enterprise Search with their applications and services. However, many developers prefer APIs that feel natural to their chosen programming language.

File upload

File upload

Did you know you can upload files directly into Kibana for quick data exploration? If you have a log file or delimited CSV, TSV, or JSON file, you can upload it, view its fields and metrics, and optionally import it into Elasticsearch.

Language Clients: Custom search solutions

Language Clients: Custom search solutions

Whether you need client application data, ECS (Elastic Common Schema)-formatted log data, server monitoring metrics, or relational database records to synchronize with Elasticsearch, you get a tailored experience with Elastic’s custom search solutions.