13 April 2018

Keeping up with Kibana: This week in Kibana for April 9, 2018

By Lee Drengenberg

Welcome to Keeping up with Kibana! This is a series of posts on new developments in the Kibana project and related learning resources and events.

While Kibana continues to evolve and improve every release, we're also dedicated to the love and nurturing of multiple release branches.  Here's a brief explanation of our policy on new features and backporting bug fixes.  This post also explains how we test Kibana.

major.minor.patch

If you look in the elastic repos in github, you see that all main products have a "master" branch, a "6.x" branch, "6.2", "5.6" and so on. 

  • The master branch is where all new works starts.  This includes new features and most bug fixes.  It might also include "breaking changes".
  • The 6.x branch gets backports of most non-breaking changes (features and bug fixes).  This is the next major.minor release.  We'll be branching this to 6.3 soon as we approach our code freeze.
  • The 5.6 and 6.2 branches contain the latest released versions (5.6.8 and 6.2.3) of those two major releases.  We only backport really important bug fixes here.  The next versions will be 5.6.9 and 6.2.4.  There's several reasons to limit changes to these branches:
    • Because the branch might contain important security fixes, we must be able to test and release it quickly.  The fewer changes we merge into these branches the less risk some functionality will regress.
    • Some users spend significant effort to do their own in-house testing of a major or minor release in a test environment before deploying to production.  We want our users to have a high level of confidence in our patch releases so that they can quickly deploy them (especially if it fixes a security vulnerability) with minimal testing.
    • Backporting takes effort to develop and test.  More time spent backporting means less time making Kibana more awesome.
  • There are older 5.5 and 6.1 branches, but they generally wouldn't have additional patch releases after 5.6 and 6.2 were released.
  • After 6.x is branched to 6.3 and released as 6.3.0, it replaces the 6.2 branch as the place where patch releases are generated.

If you followed all that, you can see why it's important to keep your stack up to date with the latest releases.  

If you're on major version 5, you should be on 5.6 and taking the latest patch releases as quickly as you can.  I know they come out often, but we only do them when really necessary.  And you should be planning to upgrade to 6+.  If you don't, then you're missing out on tons of new features and improvements!

Testing, Testing, never Resting

Unit Tests.  Of course Kibana has Unit Tests.  Doesn't everybody?  The lowest level of testing, they run on every PR on every branch.  Pretty important stuff.

System Tests.  We use Selenium to drive Kibana through it's functionality and verify the proper behavior.   These start up an Elasticsearch node and run Kibana in a way similar to how a user would run them.  These tests also run on every PR before it gets merged.  Most other products in the stack don't have a UI, which makes these tests unique to the Kibana team.

Integration Tests. These tests install the full stack on multiple platforms and run a small number of tests to make sure your Beats and Logstash data will work in Kibana with and without X-Pack features including security.  These are the top of the testing pyramid so they're a small number of tests (and require the most setup).

Manual Tests. Yes, we have them.  Why in this modern world of software and computers would we do manual testing?

  • We don't currently have our automated System Tests running on multiple browsers (only Chrome).  We're working on adding the Firefox, and IE 11, and Edge browsers.
  • Our System Tests mainly test functionality, not appearance.  So something that looks terrible could still pass the tests.  We do have some tests that compare screenshots, but the majority test for functional results.
  • Experienced human test engineers who do ad-hoc testing find new bugs, usability, or accessibility issues that the automated regression tests won't find.

Here's a functional test based on our Shakespeare example in our Getting Started guide.  It loads test data, creates an index pattern, and creates a vertical bar chart, while checking the data in the chart at each step.

ezgif-3-600d1e13a9.gif

Hope you enjoyed this peek into how we work!