ageedit
Empty values and commented lines will result in the default value, if any, being selected. If a setting is set, but not used by a given filtertype, it may generate an error.
This filtertype will iterate over the actionable list and match indices based on their age. They will remain in, or be removed from the actionable list based on the value of exclude.
Age calculationedit
units
are calculated as follows:
Unit  Seconds  Note 


 One second 

 Calculated as 60 seconds 

 Calculated as 60 minutes (60*60) 

 Calculated as 24 hours (24*60*60) 

 Calculated as 7 days (7*24*60*60) 

 Calculated as 30 days (30*24*60*60) 

 Calculated as 365 days (365*24*60*60) 
All calculations are in epoch time, which is the number of seconds elapsed since
1 Jan 1970. If no epoch
is specified in the filter, then the
current epoch timewhich is always UTCis used as the basis for comparison.
As epoch time is always increasing, lower numbers indicate dates and times in the past.
When age is calculated, unit
is multiplied by
unit_count
to obtain a total number of seconds to use as a
differential.
For example, if the time at execution were 20170407T15:00:00Z (UTC), then the
epoch timestamp would be 1491577200
. If I had an age filter defined like
this:
 filtertype: age source: creation_date direction: older unit: days unit_count: 3
The time differential would be 3*24*60*60
seconds, which is 259200
seconds.
Subtracting this value from 1491577200
gives us 1491318000
, which is
20170404T15:00:00Z (UTC), exactly 3 days in the past. The creation_date
of
indices or snapshots is compared to this timestamp. If it is older
, it stays
in the actionable list, otherwise it is removed from the actionable list.
age
filter vs. period
filter
The time differential means of calculation can lead to frustration.
Setting unit
to months
, and unit_count
to 3
will actually calculate the
age as 3*30*24*60*60
, which is 7776000
seconds. This may be a big deal. If
the date is 20170101T02:30:00Z, or 1483237800
in epoch time. Subtracting
7776000
seconds makes 1475461800
, which is 20161003T02:30:00Z. If you were
to try to match monthly indices, index2016.12
, index2016.11
, 2016.10
,
2016.09
, etc., then both index2016.09
and index2016.10
will be older
than the cutoff date. This may result in unintended behavior.
Another way this can cause issues is with weeks. Weekly indices may start on
Sunday or Monday. The age filter’s calculation doesn’t take this into
consideration, and merely tests the difference between execution time and the
timestamp on the index (from any source
).
Another means of selecting indices and snapshots is the period filter, which is perhaps a better choice for selecting weeks and months as it compensates for these differences.
name
based agesedit
Using name
as the source
tells Curator to look for a
timestring
within the index or snapshot name, and convert
that into an epoch timestamp (epoch implies UTC).
 filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: 3
A word about regular expression matching with timestrings
Timestrings are parsed from strftime patterns, like %Y.%m.%d
, into regular
expressions. For example, %Y
is 4 digits, so the regular expression for that
looks like \d{4}
, and %m
is 2 digits, so the regular expression is \d{2}
.
What this means is that a simple timestring to match year and month, %Y.%m
will result in a regular expression like this: ^.*\d{4}\.\d{2}.*$
. This
pattern will match any 4 digits, followed by a period .
, followed by 2 digits,
occurring anywhere in the index name. This means it will match monthly
indices, like index2016.12
, as well as daily indices, like
index2017.04.01
, which may not be the intended behavior.
To compensate for this, when selecting indices matching a subset of another
pattern, use a second filter with exclude
set to True
 filtertype: pattern kind: timestring value: '%Y.%m'  filtertype: pattern kind: timestring value: '%Y.%m.%d' exclude: True
This will prevent the %Y.%m
pattern from matching the %Y.%m
part of the
daily indices.
This applies whether using timestring
as a mere pattern match, or as part of
date calculations.
creation_date
based agesedit
creation_date
extracts the epoch time of index or snapshot creation.
 filtertype: age source: creation_date direction: older unit: days unit_count: 3
field_stats
based agesedit
source
can only be field_stats
when filtering indices.
In Curator 5.3 and older, source field_stats
uses the
Field Stats API
to calculate either the min_value
or the max_value
of the field
as the stats_result
, and then use that value for age
comparisons. In 5.4 and above, even though it is still called field_stats
, it
uses an aggregation to calculate the same values, as the field_stats
API is
no longer used in Elasticsearch 6.x and up.
field
must be of type date
in Elasticsearch.
 filtertype: age source: field_stats direction: older unit: days unit_count: 3 field: '@timestamp' stats_result: min_value
Required settingsedit
Dependent settingsedit

timestring (required if
source
isname
) 
field (required if
source
isfield_stats
) [Indices only] 
stats_result (only used if
source
isfield_stats
) [Indices only]
Optional settingsedit
 unit_count_pattern
 epoch

exclude (default is
False
)