24 Juli 2014 Engineering

Curator 1.2.0 Released

Von Aaron Mildenstein

Greetings! Even though it has only been a few weeks since Curator v1.1.0 was released, we're rolling out another new version. Introducing Curator v1.2.0!

New features

  • User-specified date patterns: This has long been a requested feature.
  • Support for weekly indices: This, too, has been long requested.
  • Multiple log format options: You can log in Logstash format out of the box now!

These changes are documented thoroughly in the Curator Documentation Wiki

Updates

  • Cleaner log output: Default log output is cleaner. Debug logging still shows everything.
  • Dry runs are more visible in log output: This makes it easier to follow when doing a test run.

Date patterns and --timestring

In previous releases of Curator, the date was calculated by separating the elements of the index name using a separator character. This design decision was simple for use with the Logstash indices the program was originally designed to manage. Since then, however, Curator has matured into a time-series index manager, and that has meant different index naming schemas.

There is still a need to do date math calculations by interval, and so the --time-unit option remains, though now you can also specify weeks as the time unit. The default --timestring options should still work out of the box as they did previously. They are as follows:

Time Unit « » Timestring
days %Y.%m.%d
hours %Y.%m.%d.%H
weeks %Y.%W

What this means, is that if you specify hours as your time unit, and do not specify a --timestring, the default will be %Y.%m.%d.%H, which is "Year.Month.Day.Hour" expressed in python strftime formatting. Similarly, if you were to specify weeks as your time unit, and allowed Curator to provide the default --timestring it would be %Y.%W.

Where this feature now provides value is with indices with no separator character between date elements. For example, if I had daily indices like production-20140724 you could disable the bloom filter cache for indices older than 2 days with a command like this:

curator bloom --prefix production- --older-than 2 --timestring %Y%m%d

Note that the default time unit is days in this example. Hourly indices—like hourly-2014072414 could be managed in similar fashion:

curator bloom --prefix hourly- --older-than 2 --time-unit hours --timestring %Y%m%d%H

Replacing --separator

If you were using a custom separator character with a previous version of Curator, your change should be relatively simple. If your old command was for an index like cerberus-2014-07-24, your command would have used --separator -. Now, your command will look like this:

curator delete --prefix cerberus- --older-than 30 --timestring %Y-%m-%d

Just put your separator string in between the year (%Y), month (%m), and day (%d) identifiers!

This also means that you can now do what was previously impossible in Curator: mixed separator characters. Now you could process indices like logs-2014.07.24-14 with a --timestring like %Y.%m.%d-%H.

Learn more about --timestring in the Curator Documentation Wiki.

Feedback

Many of these features came about because of user comments and requests. If you have a feature request or find a bug, please let us know!

We also love shout-outs on Twitter. Our twitter handle is @elasticsearch

Happy Curating!