I wanted to visualize the weather outside. So I built a way using some hobby hardware, Elasticsearch & Kibana.

I’m far from a meteorologist. I’m a hacker with a garage/office that I spend way too much time in. I have a bias toward things that feel like data. A friend told me that I was maybe being a bit of a garage troll; I am tucked away from the sun and warmth. I decided that I needed to figure out if she was right. This was a perfect opportunity to dive into the Internet of Playful Things: Arduino for my weather measurements, Elasticsearch for storage, and Kibana for viewing and analysis. With off-the-shelf parts, open source libraries and a Saturday afternoon available, I got to work.In true DIY blog post fashion, here’s the finished product. I’m far too impatient to test with weeks and weeks worth of data. In order to speed up my testing, I raided the freezer and bathroom for some supplies...<div class="video embed-container" style="height: 319.725px;"><img class="vidyard-player-embed" src="https://play.vidyard.com/E8MPU4HroDPfJmKKCrUFYe.jpg" data-uuid="E8MPU4HroDPfJmKKCrUFYe" data-v="4" data-type="inline" style="width: 100%;margin: auto;display: block;" width="100%"/></div> I choose the ESP8266 more and more often lately. It’s a microcontroller with WiFi capabilities that can be used by itself or with most other platforms. Folks have built compatibility layers for Node.js, Python, and Arduino with it. It has become a very popular device because of its price and capabilities. For only a couple bucks, you can add WiFi to any hobby hardware project. For about sixteen dollars, you can get a battery-powered Arduino and node/lua-compatible development board. For this project I chose a board from <a href="https://www.adafruit.com/product/2821">Adafruit</a>.Elasticsearch and Kibana provide a very tidy combination for storing and visualizing the sensor data coming out of my hardware. Prebuilt modules from Adafruit make creating your own wireless weather station simple.<h2>Things you’ll need</h2>Electronics<ul><li><a href="https://www.adafruit.com/product/2821">Adafruit Feather HUZZAH or another ESP8266 dev board</a></li><li><a href="https://www.adafruit.com/products/1603">Adafruit BMP180 Temperature/Pressure/Altitude Sensor</a></li></ul>Tools<ul><li>Soldering Iron</li><li>Hookup Wire</li></ul>Nice to have<ul><li>3.7v Lithium Battery - <a href="https://www.adafruit.com/products/1603?q=lithium&">like these</a></li><li>Breadboard - <a href="https://www.adafruit.com/products/64">half size is just right</a></li><li>Jumper Wires - <a href="https://www.adafruit.com/products/1957">like these</a></li></ul>Computer<ul><li>Arduino Software (with some extra libraries and examples, detailed below)</li><li>Elasticsearch + Kibana</li></ul>For Elasticsearch and Kibana, you can setup an instance on <a href="https://www.elastic.co/cloud/as-a-service">Elastic Cloud</a> or you’ll need to share a network with your devices and use a local instance of both.<h2>Hardware Build</h2>I recommend using a breadboard for most prototypes and temporary projects. The first step is to solder the header pins onto the board. Adafruit ships most of its modules without the pins soldered on. If you haven’t soldered before, there are lots of really good guides to getting started. SparkFun has <a href="https://learn.sparkfun.com/tutorials/how-to-solder---through-hole-soldering">a nice guide</a>. If you’re getting really into it you should dig into the <a href="http://workmanship.nasa.gov/lib/insp/2%20books/frameset.html">NASA Workmanship Standards</a>.A note on soldering and tools:Most guides on how to solder are written by people who do it a lot. Like most other hobbies and professions, using good tools will make the job easier. It’s also expensive. All you really need to get started is an iron, solder, and something to cut and strip wires with. Get better tools as you need them. My <a href="http://www.oaklandlibrary.org/locations/tool-lending-library/tool-list-lending-guidelines">local library will loan the tools</a>, yours might too. You might also have a <a href="https://wiki.hackerspaces.org/List_of_Hacker_Spaces">local hackerspace</a> which may be able to assist with tools, equipment, and techniques.My first toolset, which got me through about 10 years of experimenting.<img alt="blog-2-weather-station.jpg" data-sys-asset-uid="blt9733f1e16e3536ab" src="https://api.contentstack.io/v2/assets/57bb09cb1251899a28dc2012/download?uid=blt9733f1e16e3536ab"/>This sensor board uses SPI to communicate with the microcontroller. The ESP8266 has SPI support, so we hook up Data (SDA) and Clock (SCL) lines to the SDA and SCL lines of each board for communication.We also hook up the 3v and Ground (GND) lines to power sensor board. That’s it! 4 wires is all we need.<img alt="blog-3-weather-station.jpg" data-sys-asset-uid="blt040cf6db92ab8fcf" src="https://api.contentstack.io/v2/assets/57bb09d21251899a28dc2018/download?uid=blt040cf6db92ab8fcf"/><h2>Arduino Software</h2>If you haven’t already, download and install the Arduino IDE and ESP8266 Board Package. <a href="https://learn.adafruit.com/adafruit-feather-huzzah-esp8266/using-arduino-ide">Follow this guide if you need help</a>.You’ll also need to install the libraries below.<ul><li><a href="https://github.com/PaulStoffregen/Time">NTP</a> - NTP is the network time protocol, it’s how we get the clock day/time from the internet to generate timestamps.</li><li><a href="https://github.com/adafruit/Adafruit_Sensor">Adafruit Unified Sensor Library</a> <a href="https://learn.adafruit.com/using-the-adafruit-unified-sensor-driver/introduction">(More information)</a></li><li>Adafruit <a href="https://github.com/adafruit/Adafruit_BMP085_Unified">BMP085U</a> Sensor Library (this also covers the BMP180 sensor we are using)</li></ul><a href="https://www.arduino.cc/en/Guide/Libraries">Use this guide</a> if you need help installing Arduino Libraries.Having WiFi makes it very easy to publish directly to Elasticsearch via the HTTP interface. The only real challenge was getting the Arduino code to generate an Elasticsearch compatible timestamp. I think that we’ve got a clever and straight-forward solution, by combining the arduino `mils()` function with an NTP generated unix timestamp.Change the SSID and password in the sketch below and you should be ready to start.<h2>Configuring Elasticsearch</h2>Before we start sending data we’re going to prime Elasticsearch to index the documents we’re sending. Make sure to check the Elasticsearch URL (my `response = requests.put` line) to match your Elasticsearch endpoint.I’m using Python and the requests library, but you can use whatever tool you’re familiar with.<pre class="prettyprint">import requests import json data = { "mappings": { "reading": { "properties": { "temperature": {"type": "float"}, "pressure": {"type": "float"}, "timestamp": {"type": "date"}, }, } } } response = requests.put('http://localhost:9200/weather', data=json.dumps(data)) try: assert response.status_code is 200 except AssertionError: raise AssertionError("Your mapping was not created", response) # You could use this snippet to delete your old data if you have an error. #response = requests.delete('http://localhost:9200/weather') #try: # assert response.status_code is 200 #except AssertionError: # raise AssertionError("Your mapping was not deleted", response) </pre><h2>Sending Data</h2>Modify the <a href="#sketch1">sketch</a> as mentioned above, and upload it to your device. It should start sending data to your Elasticsearch instance. You can verify in the Serial Monitor under `Tools -&gt; Serial Monitor`. Make sure that the baud rate is set to 115200.<h2>Configuring Kibana</h2>Open your Kibana instance, click “Settings” and add a new pattern. If you type “weather” into the index pattern, it should autofill the timestamp.Mine looks like this:<img alt="blog-4-weather-station.png" data-sys-asset-uid="blt6055beb7b47ea60a" src="https://api.contentstack.io/v2/assets/57bb09d81251899a28dc201f/download?uid=blt6055beb7b47ea60a"/>Once you create a new index, you can click `Visualize` to start making graphs.Here’s one I created. Start with the settings I have in the screenshot and then experiment to find what you like.<img alt="blog-5-weather-station.png" data-sys-asset-uid="bltdcbcc5191d4eec35" src="https://api.contentstack.io/v2/assets/57bb09de7791d1a8275e38f5/download?uid=bltdcbcc5191d4eec35"/><a name="sketch1"> </a><h2>Arduino Code Listing</h2><pre class="prettyprint"> /* * Simple HTTP get webclient test */ #include &lt;ESP8266WiFi.h&gt; #include &lt;Wire.h&gt; #include &lt;Adafruit_Sensor.h&gt; #include &lt;Adafruit_BMP085_U.h&gt; #include &lt;TimeLib.h&gt; #include &lt;WiFiUdp.h&gt; Adafruit_BMP085_Unified bmp = Adafruit_BMP085_Unified(10085); static const char ntpServerName[] = "us.pool.ntp.org"; // Setup your wifi SSID and password here. const char* ssid = "CanIGetAWiFi"; const char* password = "n0youCan7"; const int timeZone = 0; // UTC // Variables needed for NTP // Elasticsearch needs us to generate timestamps for the data in order to make date histograms in Kibana. WiFiUDP Udp; unsigned int localPort = 8888; // local port to listen for UDP packets time_t getNtpTime(); void printDigits(int digits); void sendNTPpacket(IPAddress &amp;address); // This is the IP address, or DNS name of my Elasticsearch instance. const char* host = "192.168.1.215"; const int port = 9200; int motion; // Variables float temperature; String timestamp; time_t start_time; uint32_t t_ms; uint32_t start_mills; String run_mills; int milis_chars; void setup() { Serial.begin(115200); delay(100); // We start by connecting to a WiFi network Serial.println(); Serial.print("Connecting to "); Serial.println(ssid); WiFi.begin(ssid, password); while (WiFi.status() != WL_CONNECTED) { delay(500); Serial.print("."); } Serial.println(""); Serial.println("WiFi connected"); Serial.println("IP address: "); Serial.println(WiFi.localIP()); Serial.println("Setting up NTP"); Udp.begin(localPort); Serial.print("Local port: "); Serial.println(Udp.localPort()); Serial.println("waiting for sync"); setSyncProvider(getNtpTime); setSyncInterval(300); start_time = now(); Serial.println("Pressure Sensor Test"); Serial.println(""); /* Initialise the sensor */ if(!bmp.begin()) { /* There was a problem detecting the BMP085 ... check your connections */ Serial.print("Ooops, no BMP180 detected ... Check your wiring!"); while(1); } } void loop() { // Measure pressure &amp; temperature from BMP sensor // Modified from https://learn.adafruit.com/bmp085/using-the-bmp085-api-v2 sensors_event_t event; bmp.getEvent(&amp;event); float pressure = event.pressure; float temperature; bmp.getTemperature(&amp;temperature); // Use WiFiClient class to create TCP connections, connect to the Elasticsearch instance. WiFiClient client; if (!client.connect(host, port)) { Serial.println("connection failed"); return; } run_mills = String(millis()); milis_chars = run_mills.length(); // To generate a millisecond unix timestamp, we first get the second timestamp, and add to it, the last three characters of the arduino/relative millisecond timestamp timestamp = String(now()) + run_mills.charAt(milis_chars-3) + run_mills.charAt(milis_chars-2) + run_mills.charAt(milis_chars-1); // With such a simple document, we're just going to use a string to generate the JSON to send to Elasticsearch String data = "{pressure: "+String(pressure)+", temperature: "+String(temperature)+", timestamp: "+ timestamp +"}"; // We can inspect the data being sent over the Serial line, in the Arduino IDE. Serial.println(data); // We now create a URI for the request // This is the index of the Elasticsearch document we're creating String url = "/weather/reading"; // client.print(String("POST ") + url + " HTTP/1.1\r\n" + // If you're using Shield, you'll need to generate an authentication header "Content-Length: " + data.length() + "\r\n" + "\r\n" + data); // We need this delay in here to give the WiFi Time delay(50); // Read all the lines of the reply from server and print them to Serial while(client.available()){ String line = client.readStringUntil('\r'); Serial.print(line); } Serial.println(); } /* Copied from https://github.com/PaulStoffregen/Time/blob/master/examples/TimeNTP_ESP8266WiFi/TimeNTP_ESP8266WiFi.ino#L99 */ /*-------- NTP code ----------*/ const int NTP_PACKET_SIZE = 48; // NTP time is in the first 48 bytes of message byte packetBuffer[NTP_PACKET_SIZE]; //buffer to hold incoming &amp; outgoing packets time_t getNtpTime() { IPAddress ntpServerIP; // NTP server's ip address while (Udp.parsePacket() &gt; 0) ; // discard any previously received packets Serial.println("Transmit NTP Request"); // get a random server from the pool WiFi.hostByName(ntpServerName, ntpServerIP); Serial.print(ntpServerName); Serial.print(": "); Serial.println(ntpServerIP); sendNTPpacket(ntpServerIP); uint32_t beginWait = millis(); while (millis() - beginWait &lt; 1500) { int size = Udp.parsePacket(); if (size &gt;= NTP_PACKET_SIZE) { Serial.println("Receive NTP Response"); Udp.read(packetBuffer, NTP_PACKET_SIZE); // read packet into the buffer unsigned long secsSince1900; // convert four bytes starting at location 40 to a long integer secsSince1900 = (unsigned long)packetBuffer[40] &lt;&lt; 24; secsSince1900 |= (unsigned long)packetBuffer[41] &lt;&lt; 16; secsSince1900 |= (unsigned long)packetBuffer[42] &lt;&lt; 8; secsSince1900 |= (unsigned long)packetBuffer[43]; return secsSince1900 - 2208988800UL + timeZone * SECS_PER_HOUR; } } Serial.println("No NTP Response :-("); return 0; // return 0 if unable to get the time } // send an NTP request to the time server at the given address void sendNTPpacket(IPAddress &amp;address) { // set all bytes in the buffer to 0 memset(packetBuffer, 0, NTP_PACKET_SIZE); // Initialize values needed to form NTP request // (see URL above for details on the packets) packetBuffer[0] = 0b11100011; // LI, Version, Mode packetBuffer[1] = 0; // Stratum, or type of clock packetBuffer[2] = 6; // Polling Interval packetBuffer[3] = 0xEC; // Peer Clock Precision // 8 bytes of zero for Root Delay &amp; Root Dispersion packetBuffer[12] = 49; packetBuffer[13] = 0x4E; packetBuffer[14] = 49; packetBuffer[15] = 52; // all NTP fields have been given values, now // you can send a packet requesting a timestamp: Udp.beginPacket(address, 123); //NTP requests are to port 123 Udp.write(packetBuffer, NTP_PACKET_SIZE); Udp.endPacket(); } </pre><img alt="blog-6-weather-station.jpg" data-sys-asset-uid="bltcfb2a4739572e401" src="https://api.contentstack.io/v2/assets/57bb09e71251899a28dc202f/download?uid=bltcfb2a4739572e401"/>It turned out that the temperature alone wasn’t enough to dissuade my friend that I’ve been spending too much time inside the house. That being said, setting up this experiment couldn’t have been easier and I’m really looking forward to using WiFi-enabled microcontrollers to send readings to Elasticsearch for further experiments.I also got this sweet live gif of myself with a hair dryer and a popsicle on this temperature sensor, so there’s that.<img alt="blog-7-weather-station.gif" data-sys-asset-uid="bltb7b317058e3f4f13" src="https://api.contentstack.io/v2/assets/57bb0a771251899a28dc2049/download?uid=bltb7b317058e3f4f13"/><a href="https://twitter.com/issackelly"> </a><a href="https://twitter.com/issackelly">Issac Kelly</a> is an Engineer and Designer in Oakland, CA.

blog-hero-weather-station.png

blog-thumbnail-weather-station.png

An Arduino-Based Home Weather Station on the Elastic Stack

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Issac Kelly

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

En confirmant votre envoi, vous reconnaissez que vous avez lu et que vous acceptez nos <a href="/fr/legal/elastic-cloud-account-terms" target="_self">Conditions générales d'utilisation</a>, et vous acceptez de recevoir les <a href="/fr/legal/privacy-statement#how-we-use-the-information" target="_self">types de notification sélectionnés</a> à l'adresse que vous avez fournie plus haut. Pour en savoir plus ou pour revenir sur votre décision (possible à tout moment), consultez la <a href="/fr/legal/privacy-statement/" target="_self">Déclaration de confidentialité d'Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.By submitting this form, I agree to Elastic sharing my contact information with Google Cloud. I also agree to Elastic giving my information to the vendor for the purpose of shipping my food package.

GDPR Text + Google Cloud

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

En confirmant votre envoi, vous reconnaissez que vous avez lu et que vous acceptez nos <a href="/fr/legal/serverless-preview-terms" target="_self">Conditions générales d'utilisation</a>, et vous permettez à Elastic de <a href="/fr/legal/privacy-statement#how-we-use-the-information" target="_self">vous contacter</a> à propos de produits et services similaires en utilisant les informations que vous avez fournies plus haut. Pour en savoir plus ou pour revenir sur votre décision (possible à tout moment), consultez la <a href="/fr/legal/privacy-statement/" target="_self">Déclaration de confidentialité d'Elastic</a>.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

En envoyant votre demande d'inscription, vous acceptez les <a href="/fr/agreements/global/cloud-services-monthly" rel="nofollow">conditions générales d'utilisation d'Elastic Cloud</a>. Vos données personnelles seront traitées conformément à la <a href="/fr/legal/privacy-statement">déclaration de confidentialité d'Elastic</a>.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

En confirmant votre envoi, vous reconnaissez que vous avez lu et que vous acceptez nos <a href="/fr/legal/terms-of-use" target="_self">Conditions générales d'utilisation</a>, et vous permettez à Elastic de <a href="/fr/legal/privacy-statement#how-we-use-the-information" target="_self">vous contacter</a> à propos de produits et services similaires en utilisant les informations que vous avez fournies plus haut. Pour en savoir plus ou pour revenir sur votre décision (possible à tout moment), consultez la <a href="/fr/legal/privacy-statement/" target="_self">Déclaration de confidentialité d'Elastic</a>.

Explore similar demos

Overview

You'll also receive an email with related content.

You'll also receive an email with related content

Nous vous enverrons plus de contenu susceptible de vous intéresser.

Speakers

Close

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

Le contenu de cette page n'est pas disponible dans la langue sélectionnée. Chez Elastic, nous mettons tout en œuvre pour vous proposer du contenu dans différentes langues. En attendant, nous vous remercions de votre patience !

See more insights

Articles by

Author

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

Voir à quelle heure débute le webinar dans mon fuseau horaire

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

Si vous ne pouvez pas vous libérer le jour J, inscrivez-vous et nous vous enverrons l'enregistrement. Vous recevrez aussi un e-mail contenant des ressources connexes.

Small image for

On-demand webinar

Featured webinar

Register to Attend

Share on Facebook

Reset all

Upcoming webinar

View more posts

Print

Hosted by

Sign In to Attend

View next

Follow us on Facebook

Share

Highlights

Read less

Thank you for registering. We will send you a confirmation email soon.

Nous vous remercions de votre inscription. Vous allez recevoir un e-mail de confirmation.

Author

Articles by Issac Kelly

An Arduino-Based Home Weather Station on the Elastic Stack

Sign up for Elastic Cloud free trial