Sharing Saved Objectsedit

This guide describes the Sharing Saved Objects effort, and the breaking changes that plugin developers need to be aware of for the planned 8.0 release of Kibana.

Overviewedit

Saved objects (hereinafter "objects") are used to store all sorts of things in Kibana, from Dashboards to Index Patterns to Machine Learning Jobs. The effort to make objects shareable can be summarized in a single picture:

Sharing Saved Objects overview

Each plugin can register different object types to be used in Kibana. Historically, objects could be isolated (existing in a single space) or global (existing in all spaces), there was no in-between. As of the 7.12 release, Kibana now supports two additional types of objects:

Where it exists

Object IDs

Registered as:

Global

All spaces

Globally unique

namespaceType: 'agnostic'

Isolated

1 space

Unique in each space

namespaceType: 'single'

(NEW) Share-capable

1 space

Globally unique

namespaceType: 'multiple-isolated'

(NEW) Shareable

1 or more spaces

Globally unique

namespaceType: 'multiple'

Ideally, most types of objects in Kibana will eventually be shareable; however, we have also introduced share-capable objects as a stepping stone for plugin developers to fully support this feature.

Breaking changesedit

To implement this feature, we had to make a key change to how objects are serialized into raw Elasticsearch documents. As a result, some existing object IDs need to be changed, and this will cause some breaking changes to the way that consumers (plugin developers) interact with objects. We have implemented mitigations so that these changes will not affect end-users if consumers implement the required steps below.

Existing, isolated object types will need to go through a special conversion process to become share-capable upon upgrading Kibana to version 8.0. Once objects are converted, they can easily be switched to become fully shareable in any future release. This conversion will change the IDs of any existing objects that are not in the Default space. Changing object IDs itself has several knock-on effects:

  • Nonstandard links to other objects can break - mitigated by Step 1
  • "Deep link" pages (URLs) to objects can break - mitigated by Step 2 and Step 3
  • Encrypted objects may not be able to be decrypted - mitigated by Step 5

To be perfectly clear: these effects will all be mitigated if and only if you follow the steps below!

External plugins can also convert their objects, but they don’t have to do so before the 8.0 release.

Developer Flowchartedit

If you’re still reading this page, you’re probably developing a Kibana plugin that registers an object type, and you want to know what steps you need to take to prepare for the 8.0 release and mitigate any breaking changes! Depending on how you are using saved objects, you may need to take up to 5 steps, which are detailed in separate sections below. Refer to this flowchart:

Sharing Saved Objects developer flowchart

There is a proof-of-concept (POC) pull request to demonstrate these changes. It first adds a simple test plugin that allows users to create and view notes. Then, it goes through the steps of the flowchart to convert the isolated "note" objects to become share-capable. As you read this guide, you can follow along in the POC to see exactly how to take these steps.

Question 1edit

Do these objects contain links to other objects?

If your objects store any links to other objects (with an object type/ID), you need to take specific steps to ensure that these links continue functioning after the 8.0 upgrade.

Step 1edit

⚠️ This step must be completed no later than the 7.16 release. ⚠️

Ensure all object links use the root-level references field

If you answered "Yes" to Question 1, you need to make sure that your object links are only stored in the root-level references field. When a given object’s ID is changed, this field will be updated accordingly for other objects.

The image below shows two different examples of object links from a "case" object to an "action" object. The top shows the incorrect way to link to another object, and the bottom shows the correct way.

Sharing Saved Objects step 1

If your objects do not use the root-level references field, you’ll need to add a migration before the 8.0 release to fix that. Here’s a migration function for the example above:

function migrateCaseToV716(
  doc: SavedObjectUnsanitizedDoc<{ connector: { type: string; id: string } }>
): SavedObjectSanitizedDoc<unknown> {
  const {
    connector: { type: connectorType, id: connectorId, ...otherConnectorAttrs },
  } = doc.attributes;
  const { references = [] } = doc;
  return {
    ...doc,
    attributes: {
      ...doc.attributes,
      connector: otherConnectorAttrs,
    },
    references: [...references, { type: connectorType, id: connectorId, name: 'connector' }],
  };
}

...

// Use this migration function where the "case" object type is registered
migrations: {
  '7.16.0': migrateCaseToV716,
},

Reminder, don’t forget to add unit tests and integration tests!

Question 2edit

Are there any "deep links" to these objects?

A deep link is a URL to a page that shows a specific object. End-users may bookmark these URLs or schedule reports with them, so it is critical to ensure that these URLs continue working. The image below shows an example of a deep link to a Canvas workpad object:

Sharing Saved Objects deep link example

Note that some URLs may contain deep links to multiple objects, for example, a Dashboard and a filter for an Index Pattern.

Step 2edit

⚠️ This step will preferably be completed in the 7.16 release; it must be completed no later than the 8.0 release. ⚠️

Update your code to use the new SavedObjectsClient resolve() method instead of get()

If you answered "Yes" to Question 2, you need to make sure that when you use the SavedObjectsClient to fetch an object using its ID, you use a different API to do so. The existing get() function will only find an object using its current ID. To make sure your existing deep link URLs don’t break, you should use the new resolve() function; this attempts to find an object using its old ID and its current ID.

In a nutshell, if your deep link page had something like this before:

const savedObject = savedObjectsClient.get(objType, objId);

You’ll need to change it to this:

const resolveResult = savedObjectsClient.resolve(objType, objId);
const savedObject = resolveResult.saved_object;

See an example of this in step 2 of the POC!

The SavedObjectsResolveResponse interface has three fields, summarized below:

  • saved_object - The saved object that was found.
  • outcome - One of the following values: 'exactMatch' | 'aliasMatch' | 'conflict'
  • alias_target_id - This is defined if the outcome is 'aliasMatch' or 'conflict'. It means that a legacy URL alias with this ID points to an object with a different ID.

The SavedObjectsClient is available both on the server-side and the client-side. You may be fetching the object on the server-side via a custom HTTP route, or you may be fetching it on the client-side directly. Either way, the outcome and alias_target_id fields need to be passed to your client-side code, and you should update your UI accordingly in the next step.

You don’t need to use resolve() everywhere, you should only use it for deep links!

Step 3edit

⚠️ This step will preferably be completed in the 7.16 release; it must be completed no later than the 8.0 release. ⚠️

Update your client-side code to correctly handle the three different resolve() outcomes

The Spaces plugin API exposes React components and functions that you should use to render your UI in a consistent manner for end-users. Your UI will need to use the Core HTTP service and the Spaces plugin API to do this.

Your page should change according to the outcome:

Sharing Saved Objects resolve outcomes overview

See an example of this in step 3 of the POC!

  1. Update your plugin’s kibana.json to add a dependency on the Spaces plugin:

    ...
    "optionalPlugins": ["spaces"]
  2. Update your plugin’s tsconfig.json to add a dependency to the Space’s plugin’s type definitions:

    ...
    "references": [
      ...
      { "path": "../spaces/tsconfig.json" },
    ]
  3. Update your Plugin class implementation to depend on the Core HTTP service and Spaces plugin API:

    interface PluginStartDeps {
      spaces?: SpacesPluginStart;
    }
    
    export class MyPlugin implements Plugin<{}, {}, {}, PluginStartDeps> {
      public setup(core: CoreSetup<PluginStartDeps>) {
        core.application.register({
          ...
          async mount(appMountParams: AppMountParameters) {
            const [coreStart, pluginStartDeps] = await core.getStartServices();
            const { http } = coreStart;
            const { spaces: spacesApi } = pluginStartDeps;
            ...
            // pass `http` and `spacesApi` to your app when you render it
          },
        });
        ...
      }
    }
  4. In your deep link page, add a check for the 'aliasMatch' outcome:

    if (spacesApi && resolveResult.outcome === 'aliasMatch') {
      // We found this object by a legacy URL alias from its old ID; redirect the user to the page with its new ID, preserving any URL hash
      const newObjectId = resolveResult.alias_target_id!; // This is always defined if outcome === 'aliasMatch'
      const newPath = `/this/page/${newObjectId}${window.location.hash}`; // Use the *local* path within this app (do not include the "/app/appId" prefix)
      await spacesApi.ui.redirectLegacyUrl(newPath, OBJECT_NOUN);
      return;
    }

    Note that OBJECT_NOUN is optional, it just changes "object" in the toast to whatever you specify — you may want the toast to say "dashboard" or "index pattern" instead!

  5. And finally, in your deep link page, add a function that will create a callout in the case of a 'conflict' outcome:

    const getLegacyUrlConflictCallout = () => {
      // This function returns a callout component *if* we have encountered a "legacy URL conflict" scenario
      if (spacesApi && resolveResult.outcome === 'conflict') {
        // We have resolved to one object, but another object has a legacy URL alias associated with this ID/page. We should display a
        // callout with a warning for the user, and provide a way for them to navigate to the other object.
        const currentObjectId = savedObject.id;
        const otherObjectId = resolveResult.alias_target_id!; // This is always defined if outcome === 'conflict'
        const otherObjectPath = `/this/page/${otherObjectId}${window.location.hash}`; // Use the *local* path within this app (do not include the "/app/appId" prefix)
        return (
          <>
            {spacesApi.ui.components.getLegacyUrlConflict({
              objectNoun: OBJECT_NOUN,
              currentObjectId,
              otherObjectId,
              otherObjectPath,
            })}
            <EuiSpacer />
          </>
        );
      }
      return null;
    };
    ...
    return (
      <EuiPage>
        <EuiPageBody>
          <EuiPageContent>
            {/* If we have a legacy URL conflict callout to display, show it at the top of the page */}
            {getLegacyUrlConflictCallout()}
            <EuiPageContentHeader>
    ...
    );
  6. Generate staging data and test your page’s behavior with the different outcomes.

Reminder, don’t forget to add unit tests and functional tests!

Step 4edit

⚠️ This step must be completed in the 8.0 release (no earlier and no later). ⚠️

Update your server-side code to convert these objects to become "share-capable"

After Step 3 is complete, you can add the code to convert your objects.

The previous steps can be backported to the 7.x branch, but this step — the conversion itself — can only take place in 8.0! You should use a separate pull request for this.

When you register your object, you need to change the namespaceType and also add a convertToMultiNamespaceTypeVersion field. This special field will trigger the actual conversion that will take place during the Core migration upgrade process when a user installs the Kibana 8.0 release:

Sharing Saved Objects conversion code

See an example of this in step 4 of the POC!

Reminder, don’t forget to add integration tests!

Question 3edit

Are these objects encrypted?

Saved objects can optionally be encrypted by using the Encrypted Saved Objects plugin. Very few object types are encrypted, so most plugin developers will not be affected.

Step 5edit

⚠️ This step must be completed in the 8.0 release (no earlier and no later). ⚠️

Update your server-side code to add an Encrypted Saved Object (ESO) migration for these objects

If you answered "Yes" to Question 3, you need to take additional steps to make sure that your objects can still be decrypted after the conversion process. Encrypted saved objects use some fields as part of "additionally authenticated data" (AAD) to defend against different types of cryptographic attacks. The object ID is part of this AAD, and so it follows that the after the object’s ID is changed, the object will not be able to be decrypted with the standard process.

To mitigate this, you need to add a "no-op" ESO migration that will be applied immediately after the object is converted during the 8.0 upgrade process. This will decrypt the object using its old ID and then re-encrypt it using its new ID:

Sharing Saved Objects ESO migration

Reminder, don’t forget to add unit tests and integration tests!

Step 6edit

Update your code to make your objects shareable

This is not required for the 8.0 release; this additional information will be added in the near future!

Frequently asked questions (FAQ)edit

1. Why are there both "share-capable" and "shareable" object types?edit

We implemented the share-capable object type as an intermediate step for consumers who currently have isolated objects, but are not yet ready to support fully shareable objects. This is primarily because we want to make sure all object types are converted at the same time in the 8.0 release to minimize confusion and disruption for the end-user experience.

We realize that the conversion process and all that it entails can be a not-insignificant amount of work for some Kibana teams to prepare for by the 8.0 release. As long as an object is made share-capable, that ensures that its ID will be globally unique, so it will be trivial to make that object shareable later on when the time is right.

A developer can easily flip a switch to make a share-capable object into a shareable one, since these are both serialized the same way. However, we envision that each consumer will need to enact their own plan and make additional UI changes when making an object shareable. For example, some users may not have access to the Saved Objects Management page, but we still want those users to be able to see what space(s) their objects exist in and share them to other spaces. Each application should add the appropriate UI controls to handle this.

2. Why do object IDs need to be changed?edit

This is because of how isolated objects are serialized to raw Elasticsearch documents. Each raw document ID today contains its space ID (namespace) as a prefix. When objects are copied or imported to other spaces, they keep the same object ID, they just have a different prefix when they are serialized to Elasticsearch. This has resulted in a situation where many Kibana installations have saved objects in different spaces with the same object ID:

Sharing Saved Objects object ID diagram (before conversion)

Once an object is converted, we need to remove this prefix. Because of limitations with our migration process, we cannot actively check if this would result in a conflict. Therefore, we decided to pre-emptively regenerate the object ID for every object in a non-Default space to ensure that every object ID becomes globally unique:

Sharing Saved Objects object ID diagram (after conversion)

3. What if one page has deep links to multiple objects?edit

As mentioned in Question 2, some URLs may contain multiple object IDs, effectively deep linking to multiple objects. These should be handled on a case-by-case basis at the plugin owner’s discretion. A good rule of thumb is:

  • The "primary" object on the page should always handle the three resolve() outcomes as described in Step 3.
  • Any "secondary" objects on the page may handle the outcomes differently. If the secondary object ID is not important (for example, it just functions as a page anchor), it may make more sense to ignore the different outcomes. If the secondary object is important but it is not directly represented in the UI, it may make more sense to throw a descriptive error when a 'conflict' outcome is encountered.

    • Embeddables should use spacesApi.ui.components.getEmbeddableLegacyUrlConflict to render conflict errors:

      Sharing Saved Objects embeddable legacy URL conflict

      Viewing details shows the user how to disable the alias and fix the problem using the _disable_legacy_url_aliases API:

      Sharing Saved Objects embeddable legacy URL conflict (showing details)
    • If the secondary object is resolved by an external service (such as the index pattern service), the service should simply make the full outcome available to consumers.

Ideally, if a secondary object on a deep link page resolves to an 'aliasMatch' outcome, the consumer should redirect the user to a URL with the new ID and display a toast message. The reason for this is that we don’t want users relying on legacy URL aliases more often than necessary. However, such handling of secondary objects is not considered critical for the 8.0 release.

4. What is a "legacy URL alias"?edit

As depicted above, when an object is converted to become share-capable, if it exists in a non-Default space, its ID gets changed. To preserve its old ID, we also create a special object called a legacy URL alias ("alias" for short); this alias retains the target object’s old ID (sourceId), and it contains a pointer to the target object’s new ID (targetId).

Aliases are meant to be mostly invisible to end-users by design. There is no UI to manage them directly. Our vision is that aliases will be used as a stop-gap to help us through the 8.0 upgrade process, but we will nudge users away from relying on aliases so we can eventually deprecate and remove them.

5. Why are there three different resolve outcomes?edit

The resolve() function checks both if an object with the given ID exists, and if an object has an alias with the given ID.

  1. If only the former is true, the outcome is an 'exactMatch' — we found the exact object we were looking for.
  2. If only the latter is true, the outcome is an 'aliasMatch' — we found an alias with this ID, that pointed us to an object with a different ID.
  3. Finally, if both conditions are true, the outcome is a 'conflict' — we found two objects using this ID. Instead of returning an error in this situation, in the interest of usability, we decided to return the most correct match, which is the exact match. By informing the consumer that this is a conflict, the consumer can render an appropriate UI to the end-user explaining that this might not be the object they are actually looking for.

Outcome 1

When you resolve an object with its current ID, the outcome is an 'exactMatch':

Sharing Saved Objects resolve outcome 1 (exactMatch)

This can happen in the Default space and in non-Default spaces.

Outcome 2

When you resolve an object with its old ID (the ID of its alias), the outcome is an 'aliasMatch':

Sharing Saved Objects resolve outcome 2 (aliasMatch)

This outcome can only happen in non-Default spaces.

Outcome 3

The third outcome is an edge case that is a combination of the others. If you resolve an object ID and two objects are found — one as an exact match, the other as an alias match — the outcome is a 'conflict':

Sharing Saved Objects resolve outcome 3 (conflict)

We actually have controls in place to prevent this scenario from happening when you share, import, or copy objects. However, this scenario could still happen in a few different situations, if objects are created a certain way or if a user tampers with an object’s raw ES document. Since we can’t 100% rule out this scenario, we must handle it gracefully, but we do expect this will be a rare occurrence.

It is important to note that when a 'conflict' occurs, the object that is returned is the "most correct" match — the one with the ID that exactly matches.

6. Should I always use resolve instead of get?edit

Reading through this guide, you may think it is safer or better to use resolve() everywhere instead of get(). Actually, we made an explicit design decision to add a separate resolve() function because we want to limit the affects of and reliance upon legacy URL aliases. To that end, we collect anonymous usage data based on how many times resolve() is used and the different outcomes are encountered. That usage data is less useful is resolve() is used more often than necessary.

Ultimately, resolve() should only be used for data flows that involve a user-controlled deep link to an object. There is no reason to change any other data flows to use resolve().

7. What about external plugins?edit

External plugins (those not shipped with Kibana) can use this guide to convert any isolated objects to become share-capable or fully shareable! If you are an external plugin developer, the steps are the same, but you don’t need to worry about getting anything done before a specific release. The only thing you need to know is that your plugin cannot convert your objects until the 8.0 release.