Index Management | Kibana Guide [7.9]

» »

« Adding a policy to an index Ingest Node Pipelines »

Index Managementedit

Index Management enables you to view index settings, mappings, and statistics and perform index-level operations. These include refreshing, flushing, clearing the cache, force merging segments, freezing indices, and more. Practicing good index management helps ensure that your data is stored in the most cost-effective way possible.

Index Management also helps you create index templates. A template reduces the amount of bookkeeping when working with indices. Instead of manually setting up your indices, you can create them automatically from a template, ensuring that your settings, mappings, and aliases are consistently defined.

To manage your indices, open the menu, then go to Stack Management > Data > Index Management.

If security is enabled, you must have the monitor cluster privilege and the view_index_metadata and manage index privileges to view the data. For index templates, you must have the manage_index_templates cluster privilege. See Security privileges for more information.

Before using this feature, you should be familiar with index management operations. Refer to the index management APIs and the index template APIs.

View and edit indicesedit

When you open Index Management, you’re presented an overview of your configured indices. Badges indicate if an index is frozen, a follower index, or a rollup index.

Clicking a badge narrows the list to only indices of that type. You can also filter your indices using the search bar.

You can drill down into each index to investigate the index settings, mapping, and statistics. From this view, you can also edit the index settings.

Perform index-level operationsedit

Use the Manage menu to perform index-level operations. This menu is available in the index details view, or when you select the checkbox of one or more indices on the overview page. The menu includes the following actions:

Close index. Blocks the index from read/write operations. A closed index exists in the cluster, but doesn’t consume resources other than disk space. If you reopen a closed index, it goes through the normal recovery process.
Force merge index. Reduces the number of segments in your shard by merging smaller files and clearing deleted ones. Only force merge a read-only index.
Refresh index. Writes the operations in the indexing buffer to the filesystem cache. This action is automatically performed once per second. Forcing a manual refresh is useful during testing, but should not be routinely done in production because it has a performance impact.
Clear index cache. Clears all caches associated with the index.
Flush index. Frees memory by syncing the filesystem cache to disk and clearing the cache. Once the sync is complete, the internal transaction log is reset.
Freeze index. Makes the index read-only and reduces its memory footprint by moving shards to disk. Frozen indices remain searchable, but queries take longer.
Delete index. Permanently removes the index and all of its documents.
Add lifecycle policy. Specifies a policy for managing the lifecycle of the index.

Manage index templatesedit

An index template defines settings, mappings, and aliases that you can automatically apply when creating a new index. Elasticsearch applies a template to a new index based on an index pattern that matches the index name.

The Index Templates view lists your templates and enables you to examine, edit, clone, and delete them. Changes you make to an index template do not affect existing indices.

If you don’t have any templates, you can create one using the Create template wizard. Index templates are applied during index creation, so you must create the template before you create the indices.

Example: Create an index templateedit

In this example, you’ll create an index template for randomly generated log files.

Open the Create template wizard, and enter logs_template in the Name field. Set Index pattern to logstash* so the template matches any index with that index pattern. The merge order and version are both optional, and you’ll leave them blank in this example.

The second step in the Create template wizard allows you to define index settings. These settings are optional, and this example skips this step.

The logs data set requires a mapping to label the latitude and longitude pairs as geographic locations by applying the geo_point type. In the third step of the wizard, define this mapping under the Mapped fields tab as follows:

Alternatively, you can click the Load JSON link and define the mapping as JSON:

{
  "properties": {
    "geo": {
      "properties": {
        "coordinates": {
          "type": "geo_point"
        }
      }
    }
  }
}

You can create additional mapping configurations in the Dynamic templates and Advanced options tabs. No additional mappings are required for this example.

In the fourth step, define an alias named logstash.

{
  "logstash": {}
}

A summary of the template is in step 5. If everything looks right, click Create template.

At this point, you’re ready to use the Elasticsearch index API to load the logs data. In the Kibana Console, index two documents:

POST /logstash-2019.05.18/_doc
{
  "@timestamp": "2019-05-18T15:57:27.541Z",
  "ip": "225.44.217.191",
  "extension": "jpg",
  "response": "200",
  "geo": {
    "coordinates": {
      "lat": 38.53146222,
      "lon": -121.7864906
    }
  },
  "url": "https://media-for-the-masses.theacademyofperformingartsandscience.org/uploads/charles-fullerton.jpg"
}

POST /logstash-2019.05.20/_doc
{
  "@timestamp": "2019-05-20T03:44:20.844Z",
  "ip": "198.247.165.49",
  "extension": "php",
  "response": "200",
  "geo": {
    "coordinates": {
      "lat": 37.13189556,
      "lon": -76.4929875
    }
  },
  "memory": 241720,
  "url": "https://theacademyofperformingartsandscience.org/people/type:astronauts/name:laurel-b-clark/profile"
}

The mappings and alias are configured automatically based on the template. To verify, you can view one of the newly created indices using the index API.