I wanted to visualize the weather outside. So I built a way using some hobby hardware, Elasticsearch & Kibana.

I’m far from a meteorologist. I’m a hacker with a garage/office that I spend way too much time in. I have a bias toward things that feel like data. A friend told me that I was maybe being a bit of a garage troll; I am tucked away from the sun and warmth. I decided that I needed to figure out if she was right. This was a perfect opportunity to dive into the Internet of Playful Things: Arduino for my weather measurements, Elasticsearch for storage, and Kibana for viewing and analysis. With off-the-shelf parts, open source libraries and a Saturday afternoon available, I got to work.In true DIY blog post fashion, here’s the finished product. I’m far too impatient to test with weeks and weeks worth of data. In order to speed up my testing, I raided the freezer and bathroom for some supplies...<div class="video embed-container" style="height: 319.725px;"><img class="vidyard-player-embed" src="https://play.vidyard.com/E8MPU4HroDPfJmKKCrUFYe.jpg" data-uuid="E8MPU4HroDPfJmKKCrUFYe" data-v="4" data-type="inline" style="width: 100%;margin: auto;display: block;" width="100%"/></div> I choose the ESP8266 more and more often lately. It’s a microcontroller with WiFi capabilities that can be used by itself or with most other platforms. Folks have built compatibility layers for Node.js, Python, and Arduino with it. It has become a very popular device because of its price and capabilities. For only a couple bucks, you can add WiFi to any hobby hardware project. For about sixteen dollars, you can get a battery-powered Arduino and node/lua-compatible development board. For this project I chose a board from <a href="https://www.adafruit.com/product/2821">Adafruit</a>.Elasticsearch and Kibana provide a very tidy combination for storing and visualizing the sensor data coming out of my hardware. Prebuilt modules from Adafruit make creating your own wireless weather station simple.<h2>Things you’ll need</h2>Electronics<ul><li><a href="https://www.adafruit.com/product/2821">Adafruit Feather HUZZAH or another ESP8266 dev board</a></li><li><a href="https://www.adafruit.com/products/1603">Adafruit BMP180 Temperature/Pressure/Altitude Sensor</a></li></ul>Tools<ul><li>Soldering Iron</li><li>Hookup Wire</li></ul>Nice to have<ul><li>3.7v Lithium Battery - <a href="https://www.adafruit.com/products/1603?q=lithium&">like these</a></li><li>Breadboard - <a href="https://www.adafruit.com/products/64">half size is just right</a></li><li>Jumper Wires - <a href="https://www.adafruit.com/products/1957">like these</a></li></ul>Computer<ul><li>Arduino Software (with some extra libraries and examples, detailed below)</li><li>Elasticsearch + Kibana</li></ul>For Elasticsearch and Kibana, you can setup an instance on <a href="https://www.elastic.co/cloud/as-a-service">Elastic Cloud</a> or you’ll need to share a network with your devices and use a local instance of both.<h2>Hardware Build</h2>I recommend using a breadboard for most prototypes and temporary projects. The first step is to solder the header pins onto the board. Adafruit ships most of its modules without the pins soldered on. If you haven’t soldered before, there are lots of really good guides to getting started. SparkFun has <a href="https://learn.sparkfun.com/tutorials/how-to-solder---through-hole-soldering">a nice guide</a>. If you’re getting really into it you should dig into the <a href="http://workmanship.nasa.gov/lib/insp/2%20books/frameset.html">NASA Workmanship Standards</a>.A note on soldering and tools:Most guides on how to solder are written by people who do it a lot. Like most other hobbies and professions, using good tools will make the job easier. It’s also expensive. All you really need to get started is an iron, solder, and something to cut and strip wires with. Get better tools as you need them. My <a href="http://www.oaklandlibrary.org/locations/tool-lending-library/tool-list-lending-guidelines">local library will loan the tools</a>, yours might too. You might also have a <a href="https://wiki.hackerspaces.org/List_of_Hacker_Spaces">local hackerspace</a> which may be able to assist with tools, equipment, and techniques.My first toolset, which got me through about 10 years of experimenting.<img alt="blog-2-weather-station.jpg" data-sys-asset-uid="blt9733f1e16e3536ab" src="https://api.contentstack.io/v2/assets/57bb09cb1251899a28dc2012/download?uid=blt9733f1e16e3536ab"/>This sensor board uses SPI to communicate with the microcontroller. The ESP8266 has SPI support, so we hook up Data (SDA) and Clock (SCL) lines to the SDA and SCL lines of each board for communication.We also hook up the 3v and Ground (GND) lines to power sensor board. That’s it! 4 wires is all we need.<img alt="blog-3-weather-station.jpg" data-sys-asset-uid="blt040cf6db92ab8fcf" src="https://api.contentstack.io/v2/assets/57bb09d21251899a28dc2018/download?uid=blt040cf6db92ab8fcf"/><h2>Arduino Software</h2>If you haven’t already, download and install the Arduino IDE and ESP8266 Board Package. <a href="https://learn.adafruit.com/adafruit-feather-huzzah-esp8266/using-arduino-ide">Follow this guide if you need help</a>.You’ll also need to install the libraries below.<ul><li><a href="https://github.com/PaulStoffregen/Time">NTP</a> - NTP is the network time protocol, it’s how we get the clock day/time from the internet to generate timestamps.</li><li><a href="https://github.com/adafruit/Adafruit_Sensor">Adafruit Unified Sensor Library</a> <a href="https://learn.adafruit.com/using-the-adafruit-unified-sensor-driver/introduction">(More information)</a></li><li>Adafruit <a href="https://github.com/adafruit/Adafruit_BMP085_Unified">BMP085U</a> Sensor Library (this also covers the BMP180 sensor we are using)</li></ul><a href="https://www.arduino.cc/en/Guide/Libraries">Use this guide</a> if you need help installing Arduino Libraries.Having WiFi makes it very easy to publish directly to Elasticsearch via the HTTP interface. The only real challenge was getting the Arduino code to generate an Elasticsearch compatible timestamp. I think that we’ve got a clever and straight-forward solution, by combining the arduino `mils()` function with an NTP generated unix timestamp.Change the SSID and password in the sketch below and you should be ready to start.<h2>Configuring Elasticsearch</h2>Before we start sending data we’re going to prime Elasticsearch to index the documents we’re sending. Make sure to check the Elasticsearch URL (my `response = requests.put` line) to match your Elasticsearch endpoint.I’m using Python and the requests library, but you can use whatever tool you’re familiar with.<pre class="prettyprint">import requests import json data = { "mappings": { "reading": { "properties": { "temperature": {"type": "float"}, "pressure": {"type": "float"}, "timestamp": {"type": "date"}, }, } } } response = requests.put('http://localhost:9200/weather', data=json.dumps(data)) try: assert response.status_code is 200 except AssertionError: raise AssertionError("Your mapping was not created", response) # You could use this snippet to delete your old data if you have an error. #response = requests.delete('http://localhost:9200/weather') #try: # assert response.status_code is 200 #except AssertionError: # raise AssertionError("Your mapping was not deleted", response) </pre><h2>Sending Data</h2>Modify the <a href="#sketch1">sketch</a> as mentioned above, and upload it to your device. It should start sending data to your Elasticsearch instance. You can verify in the Serial Monitor under `Tools -&gt; Serial Monitor`. Make sure that the baud rate is set to 115200.<h2>Configuring Kibana</h2>Open your Kibana instance, click “Settings” and add a new pattern. If you type “weather” into the index pattern, it should autofill the timestamp.Mine looks like this:<img alt="blog-4-weather-station.png" data-sys-asset-uid="blt6055beb7b47ea60a" src="https://api.contentstack.io/v2/assets/57bb09d81251899a28dc201f/download?uid=blt6055beb7b47ea60a"/>Once you create a new index, you can click `Visualize` to start making graphs.Here’s one I created. Start with the settings I have in the screenshot and then experiment to find what you like.<img alt="blog-5-weather-station.png" data-sys-asset-uid="bltdcbcc5191d4eec35" src="https://api.contentstack.io/v2/assets/57bb09de7791d1a8275e38f5/download?uid=bltdcbcc5191d4eec35"/><a name="sketch1"> </a><h2>Arduino Code Listing</h2><pre class="prettyprint"> /* * Simple HTTP get webclient test */ #include &lt;ESP8266WiFi.h&gt; #include &lt;Wire.h&gt; #include &lt;Adafruit_Sensor.h&gt; #include &lt;Adafruit_BMP085_U.h&gt; #include &lt;TimeLib.h&gt; #include &lt;WiFiUdp.h&gt; Adafruit_BMP085_Unified bmp = Adafruit_BMP085_Unified(10085); static const char ntpServerName[] = "us.pool.ntp.org"; // Setup your wifi SSID and password here. const char* ssid = "CanIGetAWiFi"; const char* password = "n0youCan7"; const int timeZone = 0; // UTC // Variables needed for NTP // Elasticsearch needs us to generate timestamps for the data in order to make date histograms in Kibana. WiFiUDP Udp; unsigned int localPort = 8888; // local port to listen for UDP packets time_t getNtpTime(); void printDigits(int digits); void sendNTPpacket(IPAddress &amp;address); // This is the IP address, or DNS name of my Elasticsearch instance. const char* host = "192.168.1.215"; const int port = 9200; int motion; // Variables float temperature; String timestamp; time_t start_time; uint32_t t_ms; uint32_t start_mills; String run_mills; int milis_chars; void setup() { Serial.begin(115200); delay(100); // We start by connecting to a WiFi network Serial.println(); Serial.print("Connecting to "); Serial.println(ssid); WiFi.begin(ssid, password); while (WiFi.status() != WL_CONNECTED) { delay(500); Serial.print("."); } Serial.println(""); Serial.println("WiFi connected"); Serial.println("IP address: "); Serial.println(WiFi.localIP()); Serial.println("Setting up NTP"); Udp.begin(localPort); Serial.print("Local port: "); Serial.println(Udp.localPort()); Serial.println("waiting for sync"); setSyncProvider(getNtpTime); setSyncInterval(300); start_time = now(); Serial.println("Pressure Sensor Test"); Serial.println(""); /* Initialise the sensor */ if(!bmp.begin()) { /* There was a problem detecting the BMP085 ... check your connections */ Serial.print("Ooops, no BMP180 detected ... Check your wiring!"); while(1); } } void loop() { // Measure pressure &amp; temperature from BMP sensor // Modified from https://learn.adafruit.com/bmp085/using-the-bmp085-api-v2 sensors_event_t event; bmp.getEvent(&amp;event); float pressure = event.pressure; float temperature; bmp.getTemperature(&amp;temperature); // Use WiFiClient class to create TCP connections, connect to the Elasticsearch instance. WiFiClient client; if (!client.connect(host, port)) { Serial.println("connection failed"); return; } run_mills = String(millis()); milis_chars = run_mills.length(); // To generate a millisecond unix timestamp, we first get the second timestamp, and add to it, the last three characters of the arduino/relative millisecond timestamp timestamp = String(now()) + run_mills.charAt(milis_chars-3) + run_mills.charAt(milis_chars-2) + run_mills.charAt(milis_chars-1); // With such a simple document, we're just going to use a string to generate the JSON to send to Elasticsearch String data = "{pressure: "+String(pressure)+", temperature: "+String(temperature)+", timestamp: "+ timestamp +"}"; // We can inspect the data being sent over the Serial line, in the Arduino IDE. Serial.println(data); // We now create a URI for the request // This is the index of the Elasticsearch document we're creating String url = "/weather/reading"; // client.print(String("POST ") + url + " HTTP/1.1\r\n" + // If you're using Shield, you'll need to generate an authentication header "Content-Length: " + data.length() + "\r\n" + "\r\n" + data); // We need this delay in here to give the WiFi Time delay(50); // Read all the lines of the reply from server and print them to Serial while(client.available()){ String line = client.readStringUntil('\r'); Serial.print(line); } Serial.println(); } /* Copied from https://github.com/PaulStoffregen/Time/blob/master/examples/TimeNTP_ESP8266WiFi/TimeNTP_ESP8266WiFi.ino#L99 */ /*-------- NTP code ----------*/ const int NTP_PACKET_SIZE = 48; // NTP time is in the first 48 bytes of message byte packetBuffer[NTP_PACKET_SIZE]; //buffer to hold incoming &amp; outgoing packets time_t getNtpTime() { IPAddress ntpServerIP; // NTP server's ip address while (Udp.parsePacket() &gt; 0) ; // discard any previously received packets Serial.println("Transmit NTP Request"); // get a random server from the pool WiFi.hostByName(ntpServerName, ntpServerIP); Serial.print(ntpServerName); Serial.print(": "); Serial.println(ntpServerIP); sendNTPpacket(ntpServerIP); uint32_t beginWait = millis(); while (millis() - beginWait &lt; 1500) { int size = Udp.parsePacket(); if (size &gt;= NTP_PACKET_SIZE) { Serial.println("Receive NTP Response"); Udp.read(packetBuffer, NTP_PACKET_SIZE); // read packet into the buffer unsigned long secsSince1900; // convert four bytes starting at location 40 to a long integer secsSince1900 = (unsigned long)packetBuffer[40] &lt;&lt; 24; secsSince1900 |= (unsigned long)packetBuffer[41] &lt;&lt; 16; secsSince1900 |= (unsigned long)packetBuffer[42] &lt;&lt; 8; secsSince1900 |= (unsigned long)packetBuffer[43]; return secsSince1900 - 2208988800UL + timeZone * SECS_PER_HOUR; } } Serial.println("No NTP Response :-("); return 0; // return 0 if unable to get the time } // send an NTP request to the time server at the given address void sendNTPpacket(IPAddress &amp;address) { // set all bytes in the buffer to 0 memset(packetBuffer, 0, NTP_PACKET_SIZE); // Initialize values needed to form NTP request // (see URL above for details on the packets) packetBuffer[0] = 0b11100011; // LI, Version, Mode packetBuffer[1] = 0; // Stratum, or type of clock packetBuffer[2] = 6; // Polling Interval packetBuffer[3] = 0xEC; // Peer Clock Precision // 8 bytes of zero for Root Delay &amp; Root Dispersion packetBuffer[12] = 49; packetBuffer[13] = 0x4E; packetBuffer[14] = 49; packetBuffer[15] = 52; // all NTP fields have been given values, now // you can send a packet requesting a timestamp: Udp.beginPacket(address, 123); //NTP requests are to port 123 Udp.write(packetBuffer, NTP_PACKET_SIZE); Udp.endPacket(); } </pre><img alt="blog-6-weather-station.jpg" data-sys-asset-uid="bltcfb2a4739572e401" src="https://api.contentstack.io/v2/assets/57bb09e71251899a28dc202f/download?uid=bltcfb2a4739572e401"/>It turned out that the temperature alone wasn’t enough to dissuade my friend that I’ve been spending too much time inside the house. That being said, setting up this experiment couldn’t have been easier and I’m really looking forward to using WiFi-enabled microcontrollers to send readings to Elasticsearch for further experiments.I also got this sweet live gif of myself with a hair dryer and a popsicle on this temperature sensor, so there’s that.<img alt="blog-7-weather-station.gif" data-sys-asset-uid="bltb7b317058e3f4f13" src="https://api.contentstack.io/v2/assets/57bb0a771251899a28dc2049/download?uid=bltb7b317058e3f4f13"/><a href="https://twitter.com/issackelly"> </a><a href="https://twitter.com/issackelly">Issac Kelly</a> is an Engineer and Designer in Oakland, CA.

blog-hero-weather-station.png

blog-thumbnail-weather-station.png

An Arduino-Based Home Weather Station on the Elastic Stack

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Issac Kelly

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

Durch das Absenden erklären Sie, dass Sie unsere <a href="/de/legal/elastic-cloud-account-terms" target="_self">Nutzungsbedingungen</a> gelesen haben und mit ihnen einverstanden sind und dass Sie Elastic erlauben, <a href="/de/legal/privacy-statement#how-we-use-the-information" target="_self">ausgewählte Nachrichten und Informationen</a> an die oben von Ihnen bereitgestellten Kontaktangaben zu senden. Weitere Informationen, darunter auch dazu, wie Sie Ihre Einwilligung zurückziehen können, finden Sie in der <a href="/de/legal/privacy-statement/" target="_self">Datenschutzerklärung von Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

Durch das Absenden erklären Sie, dass Sie unsere <a href="/de/legal/serverless-preview-terms" target="_self">Nutzungsbedingungen</a> gelesen haben und mit ihnen einverstanden sind und dass Sie Elastic erlauben, bezüglich unserer zugehörigen Produkte und Dienstleistungen und unter Nutzung der von Ihnen oben bereitgestellten Kontaktangaben <a href="/de/legal/privacy-statement#how-we-use-the-information" target="_self">mit Ihnen in Kontakt zu treten</a>. Weitere Informationen, darunter auch dazu, wie Sie Ihre Einwilligung zurückziehen können, finden Sie in der <a href="/de/legal/privacy-statement/" target="_self">Datenschutzerklärung von Elastic</a>.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

Beim Absenden akzeptieren Sie die <a href="/de/agreements/global/cloud-services-monthly" rel="nofollow">Nutzungsbedingungen von Elastic Cloud</a>. Die Verarbeitung Ihrer personenbezogenen Daten unterliegt der <a href="/de/legal/privacy-statement">Datenschutzerklärung von Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

Durch das Absenden erklären Sie, dass Sie unsere <a href="/de/legal/terms-of-use" target="_self">Nutzungsbedingungen</a> gelesen haben und mit ihnen einverstanden sind und dass Sie Elastic erlauben, bezüglich unserer zugehörigen Produkte und Dienstleistungen und unter Nutzung der von Ihnen oben bereitgestellten Kontaktangaben <a href="/de/legal/privacy-statement#how-we-use-the-information" target="_self">mit Ihnen in Kontakt zu treten</a>. Weitere Informationen, darunter auch dazu, wie Sie Ihre Einwilligung zurückziehen können, finden Sie in der <a href="/de/legal/privacy-statement/" target="_self">Datenschutzerklärung von Elastic</a>.

Explore similar demos

Overview

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

Der Inhalt dieser Seite ist in der ausgewählten Sprache nicht verfügbar. Wir bei Elastic arbeiten daran, die bereitgestellten Inhalte in verschiedenen Sprachen anzubieten. Bis dahin bitten wir Sie um etwas Geduld und hoffen auf Ihr Verständnis!

Author

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Articles by

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Sie können nicht dabei sein? Registrieren Sie sich, damit wir Ihnen die Aufzeichnung zukommen lassen können. Außerdem erhalten Sie eine E‑Mail mit Begleitmaterial.

Small image for

On-demand webinar

Featured webinar

Register to Attend

Share on Facebook

Reset all

Upcoming webinar

View more posts

Print

Hosted by

Sign In to Attend

View next

Follow us on Facebook

Share

Highlights

Learn More

Read less

You'll also receive an email with related content

Wir schicken Ihnen zudem relevante Informationen, die von Interesse sein könnten.

Thank you for registering. We will send you a confirmation email soon.

Vielen Dank für Ihre Registrierung. Sie erhalten in Kürze eine Bestätigungs-E‑Mail von uns.

Author

Articles by Issac Kelly

An Arduino-Based Home Weather Station on the Elastic Stack

Sign up for Elastic Cloud free trial