The Treasurer handed down <a href="https://www.budget.gov.au/">Budget 2018-19 at 7:30pm on Tuesday 8 May 2018</a>. The budget is a complex set of documents and for an ordinary individual understanding or even navigating the document could be quite a daunting task. Interactive visuals have always been a preferred choice for many when it comes to deciphering numbers, particularly multi-dimensional dollar values that run across a number of years.
In this article, we will demonstrate how <a href="/products/kibana">Kibana</a> could be used to visualise a subset of the budget data elegantly than one would normally do using a spreadsheet application.
<h3>The challenge of multi-dimensional data in its raw form</h3>With data (e.g., .csv or .xls) provided in their raw form, one would normally open a spreadsheet application, select the rows and columns in a sheet, and configure some sort of visualisation. The visuals thus created would be static, and one would not be able to select a dimension or dig deeper into that column. Moreover, the process of creating the visual, sharing across an organisation or publicly becomes inefficient and error prone. And by the way, who wants to read a csv or .xls file cell by cell?
We need a simple tool to be able to visualise data and understand its content.
<h3>Visualising Australia 2018 Budget - An example</h3>Budget 2018-19 Tables and Data are publicly available <a href="https://data.gov.au/dataset/863c394c-a26c-4340-896b-a26b18af476d">here</a>.
One of the files “2018-19 Budget Dept Expenses.csv” if opened in a spreadsheet, would look like:
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt1610f65fde9f32c9/5c583e46893c9ee20b66f7ec/spreadsheet-image.jpg" data-sys-asset-uid="blt1610f65fde9f32c9" alt="spreadsheet-image.jpg">
Using Kibana, that data is now presented in an interactive manner as below:
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt6611af0b63b9bd9d/5cc353d9ee4bf29d69c58e13/australia-budget-2.png" data-sys-asset-uid="blt6611af0b63b9bd9d" alt="Australia budget data presented in Kibana" "="">
If you were interested in drilling-down on the “human services” department expenses, then just clicking on that bar would take you to a visualisation like below:
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt070b36ab0640a933/5cc35402ee4bf29d69c58e19/australia-budget-3.png" data-sys-asset-uid="blt070b36ab0640a933" alt="Drilling down into budget data with Kibana" "="">
By the way, have you noticed that these visuals are a part of a webpage and the http link could be shared with anyone you wish?
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt0ba59e988f7aec63/5c6c47ef2b22f9f161d6d356/image1.png" data-sys-asset-uid="blt0ba59e988f7aec63" alt="" "="">
We wanted to see and understand more. So we opened the file “2018-19-pbs-program-expense-line-items.csv”, and the results on piechart showed so much detail.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt564701f1ea291d17/5cc3543fee4bf29d69c58e1f/australia-budget-5.jpg" data-sys-asset-uid="blt564701f1ea291d17" alt="Detail drilldown via pie chart in Kibana" "="">
Drilling down onto one of the programs, e.g,. Employment Services, we can ascertain how much funds the budget has allocated, as depicted below:
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt0489d0d0f75e26a9/5cc3546f43283e8d640eb9bc/australia-budget-6.jpg" data-sys-asset-uid="blt0489d0d0f75e26a9" alt="Drilling into programs within the pie chart" "="">
Post the technical setup, visualisation creations and final touches, the dashboard is ready to be shared in a multi-tenant way, and it could look as simple as below:
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt024960c2a184a51e/5cc3548f887e04ba6916f953/australia-budget-7.jpg" data-sys-asset-uid="blt024960c2a184a51e" alt="The final budget dashboard in Kibana" "="">
At Elastic, we make things simple!
Now on the technical side of things...
<h3>Importing, Storing, and Visualizing Australia 2018 Budget - An Engineering Approach </h3>Steps to follow to get to the Kibana visuals that could be shared with masses.
<ul>
	<li>Sign up for an <a href="/cloud">Elastic Cloud</a> account.</li>
	<li>Download and install <a href="/downloads/logstash">Logstash</a>.</li>
	<li>Copy-paste the below configuration in your Logstash’s config path.</li>
	<li>Run the Logstash pipeline.</li>
	<li>Start creating visualisations and dashboard in Kibana.</li>
</ul>Below, you will find a Logstash configuration for importing a csv file. You may name this file as “2018-19-pbs-program-expense-line-items.conf”:
<pre class="prettyprint"># This is where your .csv file will be read
input {
 file {
 path =&gt; "/usr/share/logstash/pipeline/aubudget2018/2018-19-pbs-program-expense-line-items.csv"
 start_position =&gt; beginning
 }
}
# This section will allow you to specify the columns, their types, and any additional formatting
filter {
 dissect {
 mapping =&gt; {
 "message" =&gt; "%{Portfolio},%{Department/Agency},%{Outcome},%{Program},%{Expense type},%{Appropriation type},%{Description},%{2017-18},%{2018-19},%{2019-20},%{2020-21},%{2021-22},%{Source document},%{Source table},%{URL}"
 }
 }
 mutate {
 convert =&gt; { "2017-18" =&gt; "float" }
 convert =&gt; { "2018-19" =&gt; "float" }
 convert =&gt; { "2019-20" =&gt; "float" }
 convert =&gt; { "2020-21" =&gt; "float" }
 convert =&gt; { "2021-22" =&gt; "float" }
 }
}
# This section defines the Elasticsearch output configuration
output {
 elasticsearch {
 hosts =&gt; "https://elasticsearch-hostname:9243/"
 ssl =&gt; true
 user =&gt; "elastic"
 password =&gt; "youwishyouknewthisdidn'tyou?"
 index =&gt; "2018-19-pbs-program-expense-line-items"
 }
}
</pre>If you have a docker installed, then you could use the following command to run Logstash pipeline:
<pre class="prettyprint">docker run --rm -it -v `pwd`:/usr/share/logstash/pipeline -e xpack.monitoring.enabled=false -e log.level=debug -e pipeline.workers=4 docker.elastic.co/logstash/logstash:6.2.4
</pre>For more information on the dissect filter being used above, please <a href="/guide/en/logstash/6.2/plugins-filters-dissect.html">visit the documentation</a>.
This is it for now friends.

kibana-visualize-australia-budget-sorted-EmploymentServices-expenses-2018.png

kibana-visualize-australia-budget-sorted-expenses-thumb-2018.png

Using Elastic Stack to Explore Australia 2018 Budget

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Dr Suraj Pandey

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

El contenido de esta página no está disponible en el idioma seleccionado. Elastic está trabajando para garantizar que el contenido esté disponible en varios idiomas. Gracias por tu paciencia mientras trabajamos.

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/elastic-cloud-account-terms" target="_self">Condiciones de servicio</a>, y que acepta recibir las <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">comunicaciones seleccionadas</a> en los detalles de contacto que proporcionó anteriormente. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/serverless-preview-terms" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Al enviar esta información, aceptas los <a href="/es/agreements/global/cloud-services-monthly" rel="nofollow">Términos de servicio de Elastic</a>. Los datos personales se procesarán de conformidad con la <a href="/es/legal/privacy-statement">declaración de privacidad de Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Al enviar, confirma que ha leído y acepta nuestras <a href="/es/legal/terms-of-use" target="_self">Condiciones de servicio</a>, y que Elastic puede <a href="/es/legal/privacy-statement#how-we-use-the-information" target="_self">ponerse en contacto con usted</a> acerca de nuestros productos y servicios relacionados. Consulte la <a href="/es/legal/privacy-statement/" target="_self">Declaración de privacidad de Elastic</a> para obtener más información o para cancelar su suscripción en cualquier momento.

Hosted by

All Solutions

Follow us on Youtube

Watch now (no PT)

Articles by

Share by Email

Filters

All (no PT translation)

Contact information

See when this webinar starts in my time zone

Ver cuándo comienza este webinar en mi zona horaria

Thank you for your interest!

Register to Attend

Small image for

Register to Watch

Continue reading

Featured webinar

What to explore next...

Reset all

Load more press releases

Sign In to Attend

Share

Upcoming webinar

Learn more

Search Integrations

Share on Reddit

Share on Twitter

Share on Facebook

Load more news

View more posts

On-demand webinar

Follow us on Facebook

Author

Press Release

You'll also receive an email with related content.

You'll also receive an email with related content

Además, te enviaremos contenidos relevantes

Print

Contact Info

Overview

Video for

Read less

Register now

Share this story

See All Posts

See all top stories

View next

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

¿No puedes asistir? Regístrate y te enviaremos la grabación. También recibirás un correo electrónico con contenido relacionado.

Follow us on LinkedIn

Highlights

Explore similar demos

Follow us on Twitter

Share on LinkedIn

Load More

See more insights

Speakers

Close

Thank you for registering. We will send you a confirmation email soon.

Gracias por registrarte. Te enviaremos un correo electrónico de confirmación pronto.

Author

Articles by Suraj Pandey

Using Elastic Stack to Explore Australia 2018 Budget

Sign up for Elastic Cloud free trial