<html><head></head><body>Keeping historical data around for analysis is extremely useful, but storage costs can become a problem as the amount of stored data grows. By &quot;rolling up&quot; the data into a single, summary document in Elasticsearch, the <a href="/guide/en/elasticsearch/reference/6.5/xpack-rollup.html">rollup APIs</a> let you summarize the data and store it more compactly. You can then archive or delete the original data to save storage. When you create a rollup, you get to pick all the fields that you&#x2019;re interested in for future analysis, and a new index is created with just that rolled up data. This new rollup index then lives side by side with the index that it&#x2019;s being rolled up from. Since rollups are just like any other index &#x2014; only a lot smaller &#x2014; you already know how to query and aggregate the data inside.
Want to learn more about rolled up data? See this video from <a href="/blog/data-rollups-in-elasticsearch-you-know-for-saving-space">Elastic{ON} 2018 keynote</a> or check out our documentation on <a href="/guide/en/elasticsearch/reference/6.5/xpack-rollup.html">rolling up historical data in Elasticsearch</a>.
Version 6.5 introduces new features in Kibana for creating, starting, stopping, and deleting rollup jobs, and also provides beta features for visualizing rolled up data in a dashboard.
<h2>Getting started</h2>To check out the new features in Kibana, go to Management, and under Elasticsearch, click Rollup Jobs. If you don&#x2019;t have any existing rollup jobs, there won&#x2019;t be much to see here at first, but never fear! Click Create a rollup job to let Kibana guide you through the steps to get started.
<h2>Create a rollup job</h2>To <a href="/guide/en/kibana/current/create-and-manage-rollup-job.html">create a new rollup</a>, just fill in the name, data flow, and how often you want to roll up the data. Then define a <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#_date_histogram_2">date histogram</a> aggregation for the rollup job and optionally <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#_terms_2">terms</a>, <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#_histogram_2">histogram</a> and <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#rollup-metrics-config">metrics</a> aggregations.
Before you save the new job, Kibana will show you a summary of the rollup details so that you can double check the info before you hit save.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltd1b9105d99e15384/5c57d737a2b68bd90b9fee65/create-rollup.png" data-sys-asset-uid="bltd1b9105d99e15384" alt="" "=""><figcaption>Go to Management, and under Elasticsearch, click Rollup Jobs to create a new rollup.
</figcaption><h2>Manage your rollups</h2>Once you&#x2019;ve saved one or more rollups, you can click on the name of each job to view its details. The Manage menu in the lower right corner enables you to start, stop, and delete the rollup job.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt2bc1d17e4c54b4ae/5c57d731f967c7fc0b4c6707/manage-rollup.png" data-sys-asset-uid="blt2bc1d17e4c54b4ae" alt="" "=""><figcaption>Manage existing jobs in the new Rollup jobs view</figcaption><h3>Configuration Changes</h3>Rollup job settings can&#x2019;t be changed after the job has been created. If you want to select additional fields or redefine any terms, you would need to delete the existing job and create a new one with the updated specifications. Be sure to use a different name for the new rollup job, as reusing the same name could lead to mismatched job configurations. You can read more about <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#_logistical_details">logistical details for the rollup job configuration</a> in the Elasticsearch documentation.
<h2>Using rolled up data</h2>One way to check out these new features in Kibana is to create a rollup to capture log data from sample web logs. If you&#x2019;d like to follow along with this example, you can load the sample data set by clicking the link next to Add sample data on the Kibana home page. Click Add under Sample web logs, and then go to Management and under Elasticsearch, click Rollup Jobs to <a href="/guide/en/kibana/6.5/create-and-manage-rollup-job.html">create a new rollup</a>.
For this example, let&#x2019;s say that we want data that is older than 7 days in the target index pattern (<code>kibana_sample_data_logs</code>) to be rolled up once a day into the index <code>rollup_logstash</code>. The rolled up data can be bucketed on an hourly basis, using 60m for the time bucket configuration to allow for more granular queries, like 2h and 12h.
In the configuration below, you&#x2019;ll see that the <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#_terms_2">terms</a>, <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#_histogram_2">histogram</a>, and <a href="/guide/en/elasticsearch/reference/6.5/rollup-job-config.html#rollup-metrics-config">metrics</a> fields reflect the key information we want to retain in the rolled up data, such as where visitors are from (geo.src), what operating system they are using (machine.os.keyword), and how much data is being sent to them (bytes).
The rolled up data can now be used for analysis at a fraction of the storage cost of the original index. The original data can live side by side with the new rollup index, or be removed or archived manually or via the Index Lifecycle Management API (coming soon).
<h3>Example rollup configuration</h3><pre class="prettyprint">Name: logs_job
Index pattern: kibana_sample_data_logs
Rollup index name: rollup_logstash
Frequency: Every day at midnight
Page size: 1000
Delay (latency buffer): 7d
Date field: @timestamp
Time bucket size: 60m
Time zone: UTC
Terms: geo.src, machine.os.keyword
Histogram: bytes, memory
Histogram interval: 1000
Metrics: bytes (average)
</pre><h2>Visualize your rollup data</h2>Once you&#x2019;ve created a rollup job, you can <a href="/guide/en/kibana/current/visualize-rollup-data.html">create an index pattern and visualize the data</a> in a variety of charts, tables, maps, and more. Most visualizations support rolled up data, with the exception of Timelion, Visual Builder, and Vega visualizations (coming soon).
You can match an index pattern to rolled up data only, or mix both rolled up and raw data to visualize all data together. An index pattern can match only one rolled up index, but there is no restriction on the number of standard indices that an index pattern can match.
Combination index patterns use the same notation as other <a href="/guide/en/elasticsearch/reference/current/multi-index.html#multi-index">multiple indices</a> in Elasticsearch. To match multiple indices to create a combination index pattern, use a comma to separate the names, with no space after the comma. The notation for wildcards (<code>*</code>) and the ability to &quot;exclude&quot; (<code>-</code>) also apply (for example, <code>test*,-test3</code>). Using the information from the example rollup configuration described above, <code>rollup_logstash</code> would be used to match only the rolled up index pattern, and <code>kibana_sample_data_logs</code> would match only the index pattern for raw data. The notation for a combination index pattern with both raw and rolled up data, would be <code>rollup_logstash,kibana_sample_data_logs</code>.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt8f73c92fef597115/5c57d725c0eac4f70b57acff/combination-index-pattern.jpg" data-sys-asset-uid="blt8f73c92fef597115" alt="" "=""><figcaption>Use a combination index pattern to create a visualization that shows live data and rolled up data together.
</figcaption>You can also create dashboards that use visualizations based on rolled up and raw data.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltfafef91a6c190091/5c57d72d9cd9dcf90b7451d8/side-by-side.jpg" data-sys-asset-uid="bltfafef91a6c190091" alt="" "=""><figcaption>View side-by-side data visualizations of your rolled up data and live data in a custom Kibana dashboard.
</figcaption><h2>Wrapping up</h2>Rollups are like any other Elasticsearch index, only much smaller, so you can query and aggregate them as you would any other index. The features introduced in Kibana 6.5 provide new tools for you to create, manage, and visualize rolled up data.
You can check out the <a href="/guide/en/elasticsearch/reference/6.5/xpack-rollup.html">Elasticsearch documentation</a> to learn more about rolled up data and the <a href="/guide/en/elasticsearch/reference/6.5/rollup-api-quickref.html">rollup API</a>.
</body></html>

make-smaller.jpg

kibana-rollup-thumb.png

Create, Manage, and Visualize Rollup Data in Kibana

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Sarah Hersh

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

El contenido de esta página no está disponible en el idioma seleccionado. Elastic está trabajando para garantizar que el contenido esté disponible en varios idiomas. Gracias por tu paciencia mientras trabajamos.

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/elastic-cloud-account-terms" target="_self">Condiciones de servicio</a>, y que acepta recibir las <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">comunicaciones seleccionadas</a> en los detalles de contacto que proporcionó anteriormente. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/serverless-preview-terms" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Al enviar esta información, aceptas los <a href="/es/agreements/global/cloud-services-monthly" rel="nofollow">Términos de servicio de Elastic</a>. Los datos personales se procesarán de conformidad con la <a href="/es/legal/privacy-statement">declaración de privacidad de Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/terms-of-use" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

Hosted by

All Solutions

Follow us on Youtube

Watch now (no PT)

Articles by

Share by Email

Filters

All (no PT translation)

Contact information

See when this webinar starts in my time zone

Ver cuándo comienza este webinar en mi zona horaria

Thank you for your interest!

Register to Attend

Small image for

Register to Watch

Continue reading

Featured webinar

What to explore next...

Reset all

Load more press releases

Sign In to Attend

Share

Upcoming webinar

Learn more

Search Integrations

Share on Reddit

Share on Twitter

Share on Facebook

Load more news

View more posts

On-demand webinar

Follow us on Facebook

Author

Press Release

You'll also receive an email with related content.

You'll also receive an email with related content

Además, te enviaremos contenidos relevantes

Print

Contact Info

Overview

Video for

Read less

Register now

Share this story

See All Posts

See all top stories

View next

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

¿No puedes asistir? Regístrate y te enviaremos la grabación. También recibirás un correo electrónico con contenido relacionado.

Follow us on LinkedIn

Highlights

Explore similar demos

Follow us on Twitter

Share on LinkedIn

Load More

See more insights

Speakers

Close

Thank you for registering. We will send you a confirmation email soon.

Gracias por registrarte. Te enviaremos un correo electrónico de confirmación pronto.

Author

Articles by Sarah Hersh

Create, Manage, and Visualize Rollup Data in Kibana

Sign up for Elastic Cloud free trial