Keeping up with Kibana: This week in Kibana for October 25th, 2019

Security

Role Mappings UI

We’ve started work on a new set of screens to manage Role Mappings. We’ve only made early progress, but here’s a sneak peak of what’s to come:

Platform

New Platform

New APIs:

• We merged new core.chrome.docTitle.* APIs to replace the legacy ui/doc_title API for modifying the global document title

Other work continues on migrating UiSettings, Licensing, and SavedObjects to the New Platform. We began new work on supporting “standalone applications” to support special cases like login, logout, and status pages.

Plugin teams merged 6 migration PRs this week, 4 by the App Arch team and 2 by the Stack Services team.

Stack Services

Telemetry

We focused on preparations to move Endgame telemetry over to an Elastic endpoint and pipeline. Also explored what to do with the IP addresses stored by telemetry, how those IPs are collected and what would have to be done to remove them from cloud telemetry.

Also improved the error/failure detection process for the telemetry infrastructure. Currently working on adding aws cloudwatch alerts for telemetry lambdas.

An issue has been opened with infra to update the telemetry elastic cluster to 7.x.

A couple of watchers have been written for the telemetry indices. Currently waiting on infra to provide access so these can be pushed to staging and production.

PR for moving and renaming ui_metrics to new platform under usage_collection still in review (#47676). PR to centralized opt in service still in review as well (#48505).

Closed a big chunk of mapping issues and PRs under the telemetry repo.

Reporting

We are working on removing the once_per_server stuff from our reporting infrastructure, and trying to decouple us from the Hapi server object as much as possible.

Make-it action / Alerting

We continued working on performance enhancements for Task Manager. Some research showed setting custom refresh options other than “wait_for” on mutation operations improved performance significantly. The first step to allow this is to modify the saved objects client / repository to allow a custom refresh option on mutating operations (#48932) . The task manager enhancements PR is now using the new feature and has noticed a significant performance increase.

We worked on the alerts list UI. The list now loads from the API and supports deleting. Currently working on custom inline actions for alerts (mute, unmute, enable, disable) (#48959). Also a few new enhancement PRs have been opened on the alerting system; ability to name alerts (#49035), return usage count of each action (#49104) and the ability to tag alerts (#49320).

We continued working on the event log. Did a first pass at the usability of ECS structures for event log data and started implementing this structure.

We are working on the alert creation UI. The alerting system currently doesn’t have built in alert types. To help build the create UI, an index threshold example will be built. Had some design discussions around how to select actions when creating an alert (converting alert context to action parameters).

Operations

We have over 7,800 dependencies in Kibana. Let’s apply some Marie Kondo methods to this madness and remove those which no longer bring us joy. The next time you add a dependency, see if there is one you can remove. Each dependency has a cost to maintain and puts a burden on things like upgrading the version of Node.

Type removal

Type removal is underway to support new releases of Elasticsearch.  This has been a several step process starting a few versions back. Initially all documents were moved to the _doc and a new type field was introduced.  In some areas we used a compatibility parameter include_type. Next steps include removing usage of this parameter and type from client requests and our test cases.  #47610

Kibana’s saved objects have a type parameter that is saved in the document itself.  These will stick around, but going forward any low level elasticsearch operations should not include type.  The saved object clients will handle this automatically.

Geo-Maps App

We’ve added cardinality metric aggregation for grid source, pew pew source, and terms joins into the Maps App.  Both coordinate map and region map visualizations allow for cardinality metric aggregation. This PR closes the gap between maps application and legacy mapping visualizations.

Sometimes you need to add data to your Maps visualization and sometimes you need to remove that data; we want you to be able to be in control of what you can see.  So we’ve made it easier to remove layers from the layer action panel instead of making users open the edit panel to remove a layer.  It will be available in the next release.

App Architecture

We continue to make progress on migrating our services to the new platform.  This week we worked on input control visualization, Maps, Timelion, and data plugin services.  Along with other miscellaneous migration items, we Jest-ified more of our tests.

Explorations have started on how we could incorporate new query languages into the stack.

POC for EQL on a dashboard:

Kibana App (Visualizations, Sharing, Discover)

NP Migration

This week we dove deep into de-angularization and shimming of the various parts of kibana app (visualize, discover, dashboard, home, dev tools, Graph, …). The first phase of this is to centralize the dependencies within the apps in a central point to install a shim that passed in dependencies in a second step. This is already done for home #48618 and discover #48072 with an open PR for visualize #49293.

Installing the shim is already done for graph #47469 with PRs in progress for dev tools #49349 , dashboard #48913 and #48715 . An important PR currently blocking the shimming PRs is #48898 which adds functionality to run an app within the app/kibana route without interfering with the global angular routing.

Lens

We are focused on bug fixes and performance improvements. Please try out Lens and give us your feedback!

Elastic Charts

We are working to allow more granular customization of series, including color, tooltip visibility, and more to come (#419). We are also working on allowing fit function types to fill null data points, currently only for non-stacked, see demo below (#416).

Additionally we fixed a few bugs, relating to legend resizing with dynamic data/config (#441) another related to empty datasets(#440) and a third related to time zones and converting from luxon to momentjs for better time zone support (#436). 

Canvas

Docs Update

We added our first set of function references to the Canvas documentation. This is a highly requested addition to the documentation that explains the semantics behind expressions as well as in-depth examples. This doc currently covers roughly 40% of our existing functions with more to come.

Design

Scrollable containers

A horizontal scroll SCSS mixin has landed in EUI for better-looking scrollable containers. This complements the existing vertical scroll mixin.

Graph App

Designs for improving the Graph application are continuing to be explored and tweaked.

Canvas sidebar style tightening

Compressed form styles are being applied to the Canvas sidebar which helps the settings and style options more discoverable by users.

Data grid

The first iteration of EuiDataGrid has been merged into EUI and there is a pull request open in Kibana to make it available there. We have an immediate backlog of bugs (not too many) and feature requests (many) which we will continue to iterate against, in addition to working with various teams to implement and discover more use cases. https://elastic.github.io/eui/#/tabular-content/data-grid

QA

Release testing - 7.5.0  

Preparing Code Coverage for EAH Demo - https://github.com/elastic/kibana/pull/49247 [in progress]

Refactoring Functional Integration Tests into Kibana - https://github.com/elastic/kibana/pull/48673 [waiting on code coverage]