Singleton Command Line Interfaceedit

The curator_cli command allows users to run a single, supported action from the command-line, without needing either the client or action YAML configuration file, though it does support using the client configuration file if you want. As an important bonus, the command-line options allow you to override the settings in the curator.yml file!

While both the configuration file and the command-line arguments can be used together, it is important to note that command-line options will override file-based configuration of the same setting.

$ curator_cli --help
Usage: curator_cli [OPTIONS] COMMAND [ARGS]...

Options:
  --config PATH                   Path to configuration file.
  --hosts TEXT                    Elasticsearch URL to connect to
  --cloud_id TEXT                 Shorthand to connect to Elastic Cloud instance
  --id TEXT                       API Key "id" value
  --api_key TEXT                  API Key "api_key" value
  --username TEXT                 Username used to create "basic_auth" tuple
  --password TEXT                 Password used to create "basic_auth" tuple
  --bearer_auth TEXT
  --opaque_id TEXT
  --request_timeout FLOAT         Request timeout in seconds
  --http_compress                 Enable HTTP compression
  --verify_certs                  Verify SSL/TLS certificate(s)
  --ca_certs TEXT                 Path to CA certificate file or directory
  --client_cert TEXT              Path to client certificate file
  --client_key TEXT               Path to client certificate key
  --ssl_assert_hostname TEXT      Hostname or IP address to verify on the node's certificate.
  --ssl_assert_fingerprint TEXT   SHA-256 fingerprint of the node's certificate. If this value is given then root-of-trust
                                  verification isn't done and only the node's certificate fingerprint is verified.
  --ssl_version TEXT              Minimum acceptable TLS/SSL version
  --master-only                   Only run if the single host provided is the elected master
  --skip_version_test             Do not check the host version
  --dry-run                       Do not perform any changes.
  --loglevel [DEBUG|INFO|WARNING|ERROR|CRITICAL]
                                  Log level
  --logfile TEXT                  log file
  --logformat [default|logstash|json|ecs]
                                  Log output format
  --version                       Show the version and exit.
  --help                          Show this message and exit.

Commands:
  alias             Add/Remove Indices to/from Alias
  allocation        Shard Routing Allocation
  close             Close Indices
  delete-indices    Delete Indices
  delete-snapshots  Delete Snapshots
  forcemerge        forceMerge Indices (reduce segment count)
  open              Open Indices
  replicas          Change Replica Count
  restore           Restore Indices
  rollover          Rollover Index associated with Alias
  show-indices      Show Indices
  show-snapshots    Show Snapshots
  shrink            Shrink Indices to --number_of_shards
  snapshot          Snapshot Indices

The option flags for the given commands match those used for the same actions. The only difference is how filtering is handled.

Command-line filteringedit

Recent improvements in Curator include schema and setting validation. With these improvements, it is possible to validate filters and their many permutations if passed in a way that Curator can easily digest.

--filter_list TEXT  JSON string representing an array of filters.

This means that filters need to be passed as a single object, or an array of objects in JSON format.

Single:

--filter_list '{"filtertype":"none"}'

Multiple:

--filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":13},{"filtertype":"pattern","kind":"prefix","value":"logstash"}]'

This preserves the power of chained filters, making them available on the command line.

You may need to escape all of the double quotes on some platforms, or shells like PowerShell, for instance.

Caveats to this approach:

  1. Only one action can be taken at a time.
  2. Not all actions have singleton analogs. For example, Alias and
    Restore do not have singleton actions.

Show Indices/Snapshotsedit

One feature that the singleton command offers that the other cannot is to show which indices and snapshots are in the system. It’s a great way to visually test your filters without causing any harm to the system.

$ curator_cli show-indices --help
Usage: curator_cli show-indices [OPTIONS]

  Show indices

Options:
  --verbose           Show verbose output.
  --header            Print header if --verbose
  --epoch             Print time as epoch if --verbose
  --filter_list TEXT  JSON string representing an array of filters.
                      [required]
  --help              Show this message and exit.
$ curator_cli show-snapshots --help
Usage: curator_cli show-snapshots [OPTIONS]

  Show snapshots

Options:
  --repository TEXT   Snapshot repository name  [required]
  --filter_list TEXT  JSON string representing an array of filters.
                      [required]
  --help              Show this message and exit.

The show-snapshots command will only show snapshots matching the provided filters. The show-indices command will also do this, but also offers a few extra features.

  • --verbose adds state, total size of primary and all replicas, the document count, the number of primary and replica shards, and the creation date in ISO8601 format.
  • --header adds a header that shows the column names. This only occurs if --verbose is also selected.
  • --epoch changes the date format from ISO8601 to epoch time. If --header is also selected, the column header title will change to creation_date

There are no extra columns or --verbose output for the show-snapshots command.

Without --epoch

Index               State     Size     Docs Pri Rep   Creation Timestamp
logstash-2016.10.20 close     0.0B        0   5   1 2016-10-20T00:00:03Z
logstash-2016.10.21  open  763.3MB  5860016   5   1 2016-10-21T00:00:03Z
logstash-2016.10.22  open  759.1MB  5858450   5   1 2016-10-22T00:00:04Z
logstash-2016.10.23  open  757.8MB  5857456   5   1 2016-10-23T00:00:04Z
logstash-2016.10.24  open  771.5MB  5859720   5   1 2016-10-24T00:00:00Z
logstash-2016.10.25  open  771.0MB  5860112   5   1 2016-10-25T00:00:01Z
logstash-2016.10.27  open  658.3MB  4872830   5   1 2016-10-27T00:00:03Z
logstash-2016.10.28  open  655.1MB  5237250   5   1 2016-10-28T00:00:00Z

With --epoch

Index               State     Size     Docs Pri Rep creation_date
logstash-2016.10.20 close     0.0B        0   5   1    1476921603
logstash-2016.10.21  open  763.3MB  5860016   5   1    1477008003
logstash-2016.10.22  open  759.1MB  5858450   5   1    1477094404
logstash-2016.10.23  open  757.8MB  5857456   5   1    1477180804
logstash-2016.10.24  open  771.5MB  5859720   5   1    1477267200
logstash-2016.10.25  open  771.0MB  5860112   5   1    1477353601
logstash-2016.10.27  open  658.3MB  4872830   5   1    1477526403
logstash-2016.10.28  open  655.1MB  5237250   5   1    1477612800