Keeping up with Kibana: This week in Kibana for August 9th, 2019

Hiring

Security

PKI

Progress continues on Kibana’s PKI auth provider. The basic implementation is done, but there are some additional hurdles we still need to overcome. We’re working through the details of how the auth provider will work with Reporting, and in Cloud. Additionally, the unit and integration tests are still required.

OIDC Implicit Flow

The PR which implements the OIDC implicit flow, enabling OIDC in Cloud, merged this week. Support will be available in 7.4

Platform

New Platform 

Cleanup and polish PRs have been merged this week for the HttpService on the backend. We are now preparing HTTP to leverage the ContextService to make request handlers easier to write. This is one of the last steps before exposing the HttpService to plugins, which will allow New Platform plugins to have HTTP routes. We also merged one change to support PKI authentication for Security. 

We merged the SavedObjectService for the browser-side. Work is in progress for getting the server-side done now. There is a chance that this slips to be merged right after 7.4 FF, but we don’t expect this to be a problem since no plugins will be using this until 7.5.

The ApplicationService which allows New Platform plugins to mount UI apps has entered review. There are a few caveats which will need to be ironed out before we release any NP apps to users that use this service. Namely, there are a handful of uiExport “hacks” that need to be ported to the New Platform so some features continue to work (eg. reporting notifications) when running a NP UI app. Josh did an audit of these modules and will move to complete these before 7.5 FF.

New Platform Migration

This week teams merged 14 PRs supporting various migrations to the New Platform.

  • Kibana AppArch: 13
  • Platform: 1

Operations

Last week we have identified a problem in the plugin helpers which would prevent plugins developed against the last public version to be built. A PR was created and its about to be merged. In parallel we have also created a PR to solve a small problem with our current build in windows which was preventing us to run a full build in a windows platform. 

Finally regarding the static-fs we ran a group of tests last week to gather benchmarks which were pretty good. With those tests we found a couple of problems on windows that would be solved this week. 

Geo-Maps App

  • Switched from Mapbox Maki icons to Elastic forked version which includes additional icons that are designed to show direction when rotated. These new icons include an arrow, a boat, and a car.
  • Added filter bar to Maps application
  • GeoJSON Upload Improvements:
      1. Fixed issue causing some files to hang on indexing step
      2. Removed duplication of feature geometries that slowed down preview callback

Geo-Elastic Maps Service

More Elastic Maps Service vectors layers are in development. New layers include Romania, Albania, Bulgaria, Moldova, and Latvia should be available in production soon.

App Architecture

Data Access Services

We previously merged a PR to close the connection from the browser to Kibana in certain scenarios where a query could be safely canceled, but noticed that the connection from Kibana to ES was still not being closed. A new PR is up to correct that behavior.

We addressed a regression that caused issues with implicit value scaling of buckets in visualizations. This started a broader discussion around how we should be handling this behavior in the future.

New Platform Migration

We finished our shimming work on metric vis and tagcloud. We've been working to convert our vis expression functions to TypeScript too; this week included table vis, markdown, and vega.

After merging our conversion of Index Patterns to TypeScript, we took some time to do some cleanup and begin the process of moving the Index Patterns code to the data plugin.

We've been continuing general cleanup on the data plugin as well, while addressing miscellaneous bugs as they pop up.

Embeddables

NP-ready plugins have been merged for embeddable and embeddable_dashboard_container plugins (#40488, #41272). This brings Embeddable API closer to what it will look like in the New Platform, though we still expect that Embeddable API will change at least during 7.4 and 7.5 releases.

Other

We created @kbn/utility-types package, which gives us a mechanism for sharing TypeScript utility types across Kibana. It re-exports some utility types from the popular utility-types package, while also providing some Kibana-specific custom types.

Kibana App (Visualizations, Sharing, Discover)

Elastic Charts

Elastic Charts storybook is finally updated to version 5 with some nicer UX,UI improvements. Each story source is displayed without the storybook boilerplate that, most of the time, create confusion when looking at the example code. With that PR we also fixed some issues when rendering the charts on containers with different background colors than the standard light and dark colors.

Few fixes are applied to the tooltip to prevent: text overflows, empty tooltips and wrong value formatting because of the chart rotation.

Starting from version 9.0.0 we introduced the possibility to color a bar depending on its value: this is particularly useful with use-cases where you need to highlight a bar if the value overflows a specific threshold. 

Canvas

A nice article was published in Computerworld, and it really highlights the best of Elastic and Canvas.  We think you’ll enjoy the read.

We also have a lot of cool features cooking for 7.4!  Embeds is the word, and we’re burning the candle at both ends for this release, meaning you’ll be able to embed Canvas anywhere, and also be able to embed visualizations inside of Canvas.  Hot features, and we’re on track to deliver.

Let’s go to Monaco!

We really like slick code editors, and Monaco is the best one we’ve seen and one of the most popular.  Very slick, and Poff has been killing it with his recent updates. Follow along here, we’re clocking this in 7.4 right now.

Design

Data Grid

Work continues on EUI’s new data grid component. This week added the ability to configure which columns appear along with the ability to order them. Next up will be field sorting and and column pinning. We’re still thinking aiming to have this available for use in the 7.5 dev cycle. 

KUI is on its last legs in Kibana

KUI (Kibana UI) is the legacy styling layer that preceded EUI. We’ve been surgically removing it over the past year and think we might be able to get rid of it completely over the next minor. That would leave Bootstrap as the only legacy design dependency (mostly used in ML, Graph and Discover) within Kibana. The goal remains to be fully based on EUI by 8.0. You can track the death watch on this spreadsheet. This week we removed a big chunk of code with #42748. This is getting easier as teams have converted their UIs on their own. Thanks folks!

Some mocks for a stickied side nav

We put together some prototypes to address the request of allowing the side nav to be sticky. Although we’re still concerned this type of layout will have space problems on medium sized screens in certain pages, for folks on large resolution wide screens or those with poor eyesight it should help. Ultimately, this is a short term hotfix, while we reassess our menu needs in a world where Kibana and Cloud are more aware of each other. Look for more designs here in the next few weeks as we figure that out.

Fleet designs continue

The past few weeks we’ve been helping the ingest team work through designs for the Fleet project. We’re making good progress and have put together wireframes for that team showing off the configuration and Agent assignment systems.

Stack Services

Alerting Service

Alerts and actions are now properly space aware (#42081). The use of API keys when security is enabled is progressing well (#41389). Currently in discussion how to transfer an API key from an alert to an action via task manager (#42959). Alert throttling still in discussion (#42749).

The task manager index is configurable again (#42394). Task manager access will transition to server.plugins instead of server.taskManager soon (#42966). A flaky test was fixed causing master and 7.x CI to fail (#42418 & #42806). Task manager v2 roadmap has been created and can be found here #42055.

Work to facilitate using API keys with an SSL connection to Elasticsearch has been merged. You can now start up Elasticsearch with SSL pre-configured “yarn es snapshot --ssl” (#42527). You can now start up Kibana with an SSL connection to Elasticsearch “yarn start --ssl” (#42840). You can now make your API integration tests use SSL (#41765). This will allow alerting and actions to use proper security when executing requests at a future time on behalf of a user.

Worked started on an audit log plugin for use by actions, alerts and task manager.  Will enable writing audit records (eg, action created/updated/deleted/fired) and searching through them (for eventual ui). (#42770)

Finished the last bits on cleanup of http apis for actions (#42627)

Reporting

We’ve migrated and cleaned-up numerous RxJS control’s to TypeScript and improved our logging scenario. We’ve also adjust how reporting communicates with Chromium in order to improve our ability to generate reports in air-gapped environments. Finally, we updated our roadmap to show what and when we plan on delivering key features.

Telemetry

We opened a discuss issue for moving telemetry to OSS (#131) We also debugged missing kibana.stats in some clusters (#126)  We updated template andmappings on the production telemetry Elasticsearch.  Finally, adding telemetry opt in to the welcome screen is in review(#42110)

Localization

We automated account creation and auto population of sample data and users to the Kibana instances used to validate translations.  We also automate sending diffs to translators instead of all labels, which will make the translation process faster. (#1)