Keeping up with Kibana: This week in Kibana for December 9th, 2019
Telemetry and pulse
The new Pulse team began ramping up with telemetry this week. Pulse is the future of telemetry, which will get us more granular data about product usage and facilitate a more SaaS-like experience with all deployments of the stack.
Pulse POC
The basic framework for the Pulse POC is complete. Over the coming weeks, we'll be building on top of this framework to experiment with aspects of bi-directional communication and much more granular telemetry collection.
Kibana on Cloud
Testing: Completed automating ECE setup and building of cloud images for self-contained testing. Next, will work to implement creating deployments using ecctl and adding to CI.
Logging: Log rotation PR has been merged (#49750). Cloud already has a PR open to take advantage of this.
Docker Images: We have created an issue outlining what is needed for ESS/ECE to use the Kibana stack image by 8.0 (#52450).
Security
Previously, the idle and absolute session timeout configuration was required to be in milliseconds. This required users to perform math, and was commonly done incorrectly. We merged a PR to enable these to be specified using user-friendly durations. It's the little things that matter most!
Platform
New APIs:
- Support for configuration deprecations (#52251)
- Replacement for the context APIs in client-side code (#50231)
- This is a big one! The AppMountContext has been deprecated and replaced with a new `getStartServices` API. See the PR for an example!
- API for creating child loggers from logger instances (#52605)
- CSP config (#52698)
Fixes:
- New ESLint rule for preventing server-side code being imported into client-side code (#52447)
- Use new Licensing plugin in legacy XPack Main code (#52994)
In Progress:
- We’re working on the last work to support chromeless applications. This will unblock the migration of login, logout, and status pages to the New Platform.
- New APIs to allow applications to update their visibility and icon state (#50223) to support feature control and licensing use cases
- An *unsafe* escape hatch for opting out of request validation is in the works (#51919) - thanks Alejandro for wrapping this up before leaving for the Pulse team!
- Migrate the node UUID APIs to New Platform (#52060)
- Support for the new “enterprise” license (#52273)
- Support for the “System API” header in New Platform (#43970)
- Migrate the Elasticsearch version check to New Platform (#49785)
Stack Services
Alerting (Make It Action)
We finished the `runNow` feature for TaskManager (#51601) and added whitelist support to the PagerDuty & Slack actions (#52989). We also put up a PR to introduced a generic `schedule` field in place of `interval` on TaskManager (#52873).
We finished the alerting UI functional tests: #48959, created an action type filter for alert list: #48959, merged denormalize actionTypeId into alert action fields: #51628, and cleaned up alerting and actions feature controls for 7.6: #52286.
We merged PR "provide finer detail on action execution errors" (#52556). We also created 3 PRs that are ready to merge: "adds per-actionType enablement via config xpack.actions.enabledTypes" (#52967), "change kibana.yml xpack.actions.whitelistedHosts to default to ['*']" (#52827), and "Adds event log for actions and alerting" (#45081). We also developed an uptime/heartbeat alert-type and simulator - https://github.com/pmuellr/kbn-sample-plugins#exampleheartbeat and https://github.com/pmuellr/es-hb-sim.
We finished Alerting UI unit tests, implemented /alerting as a default tab for management UI, adjusted Create Alert UI with remove/replace not used fields, and began work on License check.
App Architecture
Examples
Data services are now exposed via a top level examples folder. Just run `yarn start --run-examples` to see them! Soon there will be an embeddable explorer as well. Anyone providing services or extension points for other developers to use can add their own examples here too!
Migration
This week we worked on the following items:
- Search bar (fully migrated) [#52622]
- ApplyFiltersPopover [#52156]
- Data plugin [#52821, #50182]
- Utils [#52744, #52500, #52615]
- Management app [#45747]
Data Access Services
- Added the timeout parameter (from the config elasticsearch.shardTimeout) to search requests, and allow overriding it in requests.
- Requests initiated from the KQL query bar for value suggestions are now cancelled when a new request is made. This should eliminate much of the extra load in large clusters when requests take longer than the time it takes to type a new character.
Bug Fixes
- Resolved an issue where changing a field format and navigating back to discover/visualizations/dashboard requires a hard page refresh in order to see the changes that were made.
- Fixed a regression with allowing plugins to register top nav menu items.
Kibana App (Visualizations, Discover, Dashboard, Graph, Elastic Charts & More)
Elastic-Charts
We fixed a few issues related to the highlighter dimensions using multiple charts on the same page and stringified values used as y value on a data array.
The PieChart PR is on its way for the review. We added the treemap chart type as a christmas gift 🎁
Migration
We completed the shimming of visualize and oss telemetry. The migration of url shortener service partially done, some remaining parts should get unblocked soon.
The deangularization and Typescriptification of ui/save_object is done and merged, same for follow up adaptions in graph. The dashboard PR ready for review.
Lens
Few bugs are fixed for 7.5.1 release (#51128, #51139, #52410)
Dashboard
The feature to add Visualization from dashboard empty screen is implemented. We are now prototyping how to add Lens visualization from dashboard:
Maps
Maintenance:
- 7.5.1: tooltips now work with Cross Cluster search, which was a regression in 7.5 (#52420)
- 7.5.1/6.8.6.: security fixes for coordinate maps and region maps required a patch release (#52309)
7.6
- User facing:
- Chores:
Elastic Maps Service
- Starting with 7.6, the URLs in Elastic Maps Service (EMS) manifests need to be relative as opposed to absolute. This will be necessary for us to provide on-premises basemaps and region boundaries in a future release.
- Also starting in 7.6, EMS releases will match the Stack release. Again, this is helpful as we will eventually move to offering on-premises EMS.
Canvas
We had a great week reviewing and moving forward some of the design work that you can read about below regarding data sources, getting started, and filtering.
- Add NP routing for custom elements - #52561
- Re-enable datemath in from/to canvas timelion args - #52159
- Allow datemath in canvas timelion function - #51918
- Add simple visual test for fullscreen - #51234
- Compress Canvas side panel inputs - #43807
- Error messages not showing - #43110
Review the list here.
Design
Team Support
We’ve started working more closely with the ML team to improve the consistency in their UI.
We’ve been working on a few user flows to pick different color palettes in Maps.
Starting with a blank Canvas is intimidating, so we’re prototyping guided tours that will make it easier for users to get started.
Also we're exploring options to improve filters in Canvas.
Working on designs for the Security team who are looking to add the ability to manage sub-feature privileges in the Roles UI.
Continuing exploration for the future of Lens and what that might mean for aggregation expressions.
EUI
TS update
EuiBasicTable and EuiInMemoryTable have been converted to typescript, providing detailed type information and better support for consumers. PR updating Kibana is open https://github.com/elastic/kibana/pull/52688
Form Guidelines
A PR is up for adding Form Guidelines to the EUI docs. We are still working through much of it, but will be a nice addition.
More color palettes
We’re adjusting the color blind palette to work in both light and dark modes as well as add more predefined color palettes for quantitative charts. The functions will also let you pass in the number of steps and interpolate dynamically.
Kibana navigation
We’re moving forward with nav grouping and have buy-in from all stakeholders. Development has started.
Operations
New
- Public documentation for max-old-space-size (#52310)
- Print out agent debugging links during CI to inspect resource metrics (CPU/Memory/Disk IO). (#52812)
Bug Fixes
- “@elasticmachine merge upstream” can now be surrounded by whitespace (elasticsearch-infra#41)