I wanted to visualize the weather outside. So I built a way using some hobby hardware, Elasticsearch & Kibana.

I’m far from a meteorologist. I’m a hacker with a garage/office that I spend way too much time in. I have a bias toward things that feel like data. A friend told me that I was maybe being a bit of a garage troll; I am tucked away from the sun and warmth. I decided that I needed to figure out if she was right. This was a perfect opportunity to dive into the Internet of Playful Things: Arduino for my weather measurements, Elasticsearch for storage, and Kibana for viewing and analysis. With off-the-shelf parts, open source libraries and a Saturday afternoon available, I got to work.In true DIY blog post fashion, here’s the finished product. I’m far too impatient to test with weeks and weeks worth of data. In order to speed up my testing, I raided the freezer and bathroom for some supplies...<div class="video embed-container" style="height: 319.725px;"><img class="vidyard-player-embed" src="https://play.vidyard.com/E8MPU4HroDPfJmKKCrUFYe.jpg" data-uuid="E8MPU4HroDPfJmKKCrUFYe" data-v="4" data-type="inline" style="width: 100%;margin: auto;display: block;" width="100%"/></div> I choose the ESP8266 more and more often lately. It’s a microcontroller with WiFi capabilities that can be used by itself or with most other platforms. Folks have built compatibility layers for Node.js, Python, and Arduino with it. It has become a very popular device because of its price and capabilities. For only a couple bucks, you can add WiFi to any hobby hardware project. For about sixteen dollars, you can get a battery-powered Arduino and node/lua-compatible development board. For this project I chose a board from <a href="https://www.adafruit.com/product/2821">Adafruit</a>.Elasticsearch and Kibana provide a very tidy combination for storing and visualizing the sensor data coming out of my hardware. Prebuilt modules from Adafruit make creating your own wireless weather station simple.<h2>Things you’ll need</h2>Electronics<ul><li><a href="https://www.adafruit.com/product/2821">Adafruit Feather HUZZAH or another ESP8266 dev board</a></li><li><a href="https://www.adafruit.com/products/1603">Adafruit BMP180 Temperature/Pressure/Altitude Sensor</a></li></ul>Tools<ul><li>Soldering Iron</li><li>Hookup Wire</li></ul>Nice to have<ul><li>3.7v Lithium Battery - <a href="https://www.adafruit.com/products/1603?q=lithium&">like these</a></li><li>Breadboard - <a href="https://www.adafruit.com/products/64">half size is just right</a></li><li>Jumper Wires - <a href="https://www.adafruit.com/products/1957">like these</a></li></ul>Computer<ul><li>Arduino Software (with some extra libraries and examples, detailed below)</li><li>Elasticsearch + Kibana</li></ul>For Elasticsearch and Kibana, you can setup an instance on <a href="https://www.elastic.co/cloud/as-a-service">Elastic Cloud</a> or you’ll need to share a network with your devices and use a local instance of both.<h2>Hardware Build</h2>I recommend using a breadboard for most prototypes and temporary projects. The first step is to solder the header pins onto the board. Adafruit ships most of its modules without the pins soldered on. If you haven’t soldered before, there are lots of really good guides to getting started. SparkFun has <a href="https://learn.sparkfun.com/tutorials/how-to-solder---through-hole-soldering">a nice guide</a>. If you’re getting really into it you should dig into the <a href="http://workmanship.nasa.gov/lib/insp/2%20books/frameset.html">NASA Workmanship Standards</a>.A note on soldering and tools:Most guides on how to solder are written by people who do it a lot. Like most other hobbies and professions, using good tools will make the job easier. It’s also expensive. All you really need to get started is an iron, solder, and something to cut and strip wires with. Get better tools as you need them. My <a href="http://www.oaklandlibrary.org/locations/tool-lending-library/tool-list-lending-guidelines">local library will loan the tools</a>, yours might too. You might also have a <a href="https://wiki.hackerspaces.org/List_of_Hacker_Spaces">local hackerspace</a> which may be able to assist with tools, equipment, and techniques.My first toolset, which got me through about 10 years of experimenting.<img alt="blog-2-weather-station.jpg" data-sys-asset-uid="blt9733f1e16e3536ab" src="https://api.contentstack.io/v2/assets/57bb09cb1251899a28dc2012/download?uid=blt9733f1e16e3536ab"/>This sensor board uses SPI to communicate with the microcontroller. The ESP8266 has SPI support, so we hook up Data (SDA) and Clock (SCL) lines to the SDA and SCL lines of each board for communication.We also hook up the 3v and Ground (GND) lines to power sensor board. That’s it! 4 wires is all we need.<img alt="blog-3-weather-station.jpg" data-sys-asset-uid="blt040cf6db92ab8fcf" src="https://api.contentstack.io/v2/assets/57bb09d21251899a28dc2018/download?uid=blt040cf6db92ab8fcf"/><h2>Arduino Software</h2>If you haven’t already, download and install the Arduino IDE and ESP8266 Board Package. <a href="https://learn.adafruit.com/adafruit-feather-huzzah-esp8266/using-arduino-ide">Follow this guide if you need help</a>.You’ll also need to install the libraries below.<ul><li><a href="https://github.com/PaulStoffregen/Time">NTP</a> - NTP is the network time protocol, it’s how we get the clock day/time from the internet to generate timestamps.</li><li><a href="https://github.com/adafruit/Adafruit_Sensor">Adafruit Unified Sensor Library</a> <a href="https://learn.adafruit.com/using-the-adafruit-unified-sensor-driver/introduction">(More information)</a></li><li>Adafruit <a href="https://github.com/adafruit/Adafruit_BMP085_Unified">BMP085U</a> Sensor Library (this also covers the BMP180 sensor we are using)</li></ul><a href="https://www.arduino.cc/en/Guide/Libraries">Use this guide</a> if you need help installing Arduino Libraries.Having WiFi makes it very easy to publish directly to Elasticsearch via the HTTP interface. The only real challenge was getting the Arduino code to generate an Elasticsearch compatible timestamp. I think that we’ve got a clever and straight-forward solution, by combining the arduino `mils()` function with an NTP generated unix timestamp.Change the SSID and password in the sketch below and you should be ready to start.<h2>Configuring Elasticsearch</h2>Before we start sending data we’re going to prime Elasticsearch to index the documents we’re sending. Make sure to check the Elasticsearch URL (my `response = requests.put` line) to match your Elasticsearch endpoint.I’m using Python and the requests library, but you can use whatever tool you’re familiar with.<pre class="prettyprint">import requests import json data = { "mappings": { "reading": { "properties": { "temperature": {"type": "float"}, "pressure": {"type": "float"}, "timestamp": {"type": "date"}, }, } } } response = requests.put('http://localhost:9200/weather', data=json.dumps(data)) try: assert response.status_code is 200 except AssertionError: raise AssertionError("Your mapping was not created", response) # You could use this snippet to delete your old data if you have an error. #response = requests.delete('http://localhost:9200/weather') #try: # assert response.status_code is 200 #except AssertionError: # raise AssertionError("Your mapping was not deleted", response) </pre><h2>Sending Data</h2>Modify the <a href="#sketch1">sketch</a> as mentioned above, and upload it to your device. It should start sending data to your Elasticsearch instance. You can verify in the Serial Monitor under `Tools -&gt; Serial Monitor`. Make sure that the baud rate is set to 115200.<h2>Configuring Kibana</h2>Open your Kibana instance, click “Settings” and add a new pattern. If you type “weather” into the index pattern, it should autofill the timestamp.Mine looks like this:<img alt="blog-4-weather-station.png" data-sys-asset-uid="blt6055beb7b47ea60a" src="https://api.contentstack.io/v2/assets/57bb09d81251899a28dc201f/download?uid=blt6055beb7b47ea60a"/>Once you create a new index, you can click `Visualize` to start making graphs.Here’s one I created. Start with the settings I have in the screenshot and then experiment to find what you like.<img alt="blog-5-weather-station.png" data-sys-asset-uid="bltdcbcc5191d4eec35" src="https://api.contentstack.io/v2/assets/57bb09de7791d1a8275e38f5/download?uid=bltdcbcc5191d4eec35"/><a name="sketch1"> </a><h2>Arduino Code Listing</h2><pre class="prettyprint"> /* * Simple HTTP get webclient test */ #include &lt;ESP8266WiFi.h&gt; #include &lt;Wire.h&gt; #include &lt;Adafruit_Sensor.h&gt; #include &lt;Adafruit_BMP085_U.h&gt; #include &lt;TimeLib.h&gt; #include &lt;WiFiUdp.h&gt; Adafruit_BMP085_Unified bmp = Adafruit_BMP085_Unified(10085); static const char ntpServerName[] = "us.pool.ntp.org"; // Setup your wifi SSID and password here. const char* ssid = "CanIGetAWiFi"; const char* password = "n0youCan7"; const int timeZone = 0; // UTC // Variables needed for NTP // Elasticsearch needs us to generate timestamps for the data in order to make date histograms in Kibana. WiFiUDP Udp; unsigned int localPort = 8888; // local port to listen for UDP packets time_t getNtpTime(); void printDigits(int digits); void sendNTPpacket(IPAddress &amp;address); // This is the IP address, or DNS name of my Elasticsearch instance. const char* host = "192.168.1.215"; const int port = 9200; int motion; // Variables float temperature; String timestamp; time_t start_time; uint32_t t_ms; uint32_t start_mills; String run_mills; int milis_chars; void setup() { Serial.begin(115200); delay(100); // We start by connecting to a WiFi network Serial.println(); Serial.print("Connecting to "); Serial.println(ssid); WiFi.begin(ssid, password); while (WiFi.status() != WL_CONNECTED) { delay(500); Serial.print("."); } Serial.println(""); Serial.println("WiFi connected"); Serial.println("IP address: "); Serial.println(WiFi.localIP()); Serial.println("Setting up NTP"); Udp.begin(localPort); Serial.print("Local port: "); Serial.println(Udp.localPort()); Serial.println("waiting for sync"); setSyncProvider(getNtpTime); setSyncInterval(300); start_time = now(); Serial.println("Pressure Sensor Test"); Serial.println(""); /* Initialise the sensor */ if(!bmp.begin()) { /* There was a problem detecting the BMP085 ... check your connections */ Serial.print("Ooops, no BMP180 detected ... Check your wiring!"); while(1); } } void loop() { // Measure pressure &amp; temperature from BMP sensor // Modified from https://learn.adafruit.com/bmp085/using-the-bmp085-api-v2 sensors_event_t event; bmp.getEvent(&amp;event); float pressure = event.pressure; float temperature; bmp.getTemperature(&amp;temperature); // Use WiFiClient class to create TCP connections, connect to the Elasticsearch instance. WiFiClient client; if (!client.connect(host, port)) { Serial.println("connection failed"); return; } run_mills = String(millis()); milis_chars = run_mills.length(); // To generate a millisecond unix timestamp, we first get the second timestamp, and add to it, the last three characters of the arduino/relative millisecond timestamp timestamp = String(now()) + run_mills.charAt(milis_chars-3) + run_mills.charAt(milis_chars-2) + run_mills.charAt(milis_chars-1); // With such a simple document, we're just going to use a string to generate the JSON to send to Elasticsearch String data = "{pressure: "+String(pressure)+", temperature: "+String(temperature)+", timestamp: "+ timestamp +"}"; // We can inspect the data being sent over the Serial line, in the Arduino IDE. Serial.println(data); // We now create a URI for the request // This is the index of the Elasticsearch document we're creating String url = "/weather/reading"; // client.print(String("POST ") + url + " HTTP/1.1\r\n" + // If you're using Shield, you'll need to generate an authentication header "Content-Length: " + data.length() + "\r\n" + "\r\n" + data); // We need this delay in here to give the WiFi Time delay(50); // Read all the lines of the reply from server and print them to Serial while(client.available()){ String line = client.readStringUntil('\r'); Serial.print(line); } Serial.println(); } /* Copied from https://github.com/PaulStoffregen/Time/blob/master/examples/TimeNTP_ESP8266WiFi/TimeNTP_ESP8266WiFi.ino#L99 */ /*-------- NTP code ----------*/ const int NTP_PACKET_SIZE = 48; // NTP time is in the first 48 bytes of message byte packetBuffer[NTP_PACKET_SIZE]; //buffer to hold incoming &amp; outgoing packets time_t getNtpTime() { IPAddress ntpServerIP; // NTP server's ip address while (Udp.parsePacket() &gt; 0) ; // discard any previously received packets Serial.println("Transmit NTP Request"); // get a random server from the pool WiFi.hostByName(ntpServerName, ntpServerIP); Serial.print(ntpServerName); Serial.print(": "); Serial.println(ntpServerIP); sendNTPpacket(ntpServerIP); uint32_t beginWait = millis(); while (millis() - beginWait &lt; 1500) { int size = Udp.parsePacket(); if (size &gt;= NTP_PACKET_SIZE) { Serial.println("Receive NTP Response"); Udp.read(packetBuffer, NTP_PACKET_SIZE); // read packet into the buffer unsigned long secsSince1900; // convert four bytes starting at location 40 to a long integer secsSince1900 = (unsigned long)packetBuffer[40] &lt;&lt; 24; secsSince1900 |= (unsigned long)packetBuffer[41] &lt;&lt; 16; secsSince1900 |= (unsigned long)packetBuffer[42] &lt;&lt; 8; secsSince1900 |= (unsigned long)packetBuffer[43]; return secsSince1900 - 2208988800UL + timeZone * SECS_PER_HOUR; } } Serial.println("No NTP Response :-("); return 0; // return 0 if unable to get the time } // send an NTP request to the time server at the given address void sendNTPpacket(IPAddress &amp;address) { // set all bytes in the buffer to 0 memset(packetBuffer, 0, NTP_PACKET_SIZE); // Initialize values needed to form NTP request // (see URL above for details on the packets) packetBuffer[0] = 0b11100011; // LI, Version, Mode packetBuffer[1] = 0; // Stratum, or type of clock packetBuffer[2] = 6; // Polling Interval packetBuffer[3] = 0xEC; // Peer Clock Precision // 8 bytes of zero for Root Delay &amp; Root Dispersion packetBuffer[12] = 49; packetBuffer[13] = 0x4E; packetBuffer[14] = 49; packetBuffer[15] = 52; // all NTP fields have been given values, now // you can send a packet requesting a timestamp: Udp.beginPacket(address, 123); //NTP requests are to port 123 Udp.write(packetBuffer, NTP_PACKET_SIZE); Udp.endPacket(); } </pre><img alt="blog-6-weather-station.jpg" data-sys-asset-uid="bltcfb2a4739572e401" src="https://api.contentstack.io/v2/assets/57bb09e71251899a28dc202f/download?uid=bltcfb2a4739572e401"/>It turned out that the temperature alone wasn’t enough to dissuade my friend that I’ve been spending too much time inside the house. That being said, setting up this experiment couldn’t have been easier and I’m really looking forward to using WiFi-enabled microcontrollers to send readings to Elasticsearch for further experiments.I also got this sweet live gif of myself with a hair dryer and a popsicle on this temperature sensor, so there’s that.<img alt="blog-7-weather-station.gif" data-sys-asset-uid="bltb7b317058e3f4f13" src="https://api.contentstack.io/v2/assets/57bb0a771251899a28dc2049/download?uid=bltb7b317058e3f4f13"/><a href="https://twitter.com/issackelly"> </a><a href="https://twitter.com/issackelly">Issac Kelly</a> is an Engineer and Designer in Oakland, CA.

blog-hero-weather-station.png

blog-thumbnail-weather-station.png

An Arduino-Based Home Weather Station on the Elastic Stack

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Issac Kelly

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

제출하면 귀하가 <a href="/kr/legal/elastic-cloud-account-terms" target="_self">Elastic 서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 연락처로 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">선택한 커뮤니케이션</a>을 수신하는 데 동의한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

제출하면 귀하가 Elastic <a href="/kr/legal/serverless-preview-terms" target="_self">서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 세부 정보를 사용하여 Elastic이 관련 제품 및 서비스에 대해 귀하에게 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">연락할 수 있음</a>을 이해한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

제출하면 <a href="/kr/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud 서비스 약관</a>에 동의하시게 됩니다. 개인 데이터는 <a href="/kr/legal/privacy-statement">Elastic의 개인정보 취급방침</a>에 따라 처리됩니다.

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

제출하면 귀하가 Elastic <a href="/kr/legal/terms-of-use" target="_self">서비스 약관</a>을 읽고 이에 동의하며, 위에 제공된 세부 정보를 사용하여 Elastic이 관련 제품 및 서비스에 대해 귀하에게 <a href="/kr/legal/privacy-statement#how-we-use-the-information" target="_self">연락할 수 있음</a>을 이해한 것으로 간주됩니다. 언제든지 수신을 거부할 수 있으며 자세한 내용은 <a href="/kr/legal/privacy-statement/" target="_self">Elastic 개인정보 취급방침</a>을 참조하시기 바랍니다.

Explore similar demos

Overview

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

이 페이지의 콘텐츠는 선택하신 언어로 제공되지 않습니다. Elastic은 다양한 언어로 콘텐츠를 제공하기 위해 최선을 다하고 있습니다.조금만 더 기다려주세요!

Author

Articles by

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

내 표준 시간대로 웨비나 시작 시간 확인하기

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

참여가 어려우신가요? 등록하시면 녹화 자료를 보내드리겠습니다. 이메일로 관련 콘텐츠도 받아보실 수 있습니다.

Small image for

On-demand webinar

Featured webinar

Register to Attend

Share on Facebook

Reset all

Upcoming webinar

View more posts

Print

Hosted by

Sign In to Attend

View next

Follow us on Facebook

Share

Highlights

Read less

You'll also receive an email with related content

이메일을 통해 관련 내용을 보내드리겠습니다.

Thank you for registering. We will send you a confirmation email soon.

등록해 주셔서 감사합니다. 곧 확인 이메일을 보내드리겠습니다.

Author

Articles by Issac Kelly

An Arduino-Based Home Weather Station on the Elastic Stack

Sign up for Elastic Cloud free trial