period
editperiod
editThis filtertype will iterate over the actionable list and match indices or snapshots based on whether they fit within the given time range. They will remain in, or be removed from the actionable list based on the value of exclude.
- filtertype: period period_type: relative source: name range_from: -1 range_to: -1 timestring: '%Y.%m.%d' unit: weeks week_starts_on: sunday
Empty values and commented lines will result in the default value, if any, being selected. If a setting is set, but not used by a given filtertype, it may generate an error.
Relative Periods
editA relative period will be reckoned relative to execution time, unless an
epoch timestamp is provided. Reckoning is truncated to the most
recent whole unit, where a unit can be one of hours
, days
, weeks
,
months
, or years
. For example, if I selected hours
as my unit
, and I
began execution at 02:35, then the point of reckoning would be 02:00. This is
relatively easy with days
, months
, and years
, but slightly more complicated
with weeks
. Some users may wish to reckon weeks by the ISO standard, which
starts weeks on Monday. Others may wish to use Sunday as the first day of the
week. Both are acceptable options with the period
filter. The default behavior
for weeks
is to have Sunday be the start of the week. This can be overridden
with week_starts_on as follows:
- filtertype: period period_type: relative source: name range_from: -1 range_to: -1 timestring: '%Y.%m.%d' unit: weeks week_starts_on: monday
range_from and range_to are counters of whole
units. A negative number indicates a whole unit in the past, while
a positive number indicates a whole unit in the future. A 0
indicates the
present unit. With such a timeline mentality, it is relatively easy to create
a date range that will meet your needs.
If the time of execution time is 2017-04-03T13:45:23.831, this table will help you figure out what the previous whole unit, current unit, and next whole unit will be, in ISO8601 format.
unit | -1 | 0 | +1 |
---|---|---|---|
hours |
2017-04-03T12:00:00 |
2017-04-03T13:00:00 |
2017-04-03T14:00:00 |
days |
2017-04-02T00:00:00 |
2017-04-03T00:00:00 |
2017-04-04T00:00:00 |
weeks sun |
2017-03-26T00:00:00 |
2017-04-02T00:00:00 |
2017-04-09T00:00:00 |
weeks mon |
2017-03-27T00:00:00 |
2017-04-03T00:00:00 |
2017-04-10T00:00:00 |
months |
2017-03-01T00:00:00 |
2017-04-01T00:00:00 |
2017-05-01T00:00:00 |
years |
2016-01-01T00:00:00 |
2017-01-01T00:00:00 |
2018-01-01T00:00:00 |
Ranges must be from older dates to newer dates, or smaller numbers (including negative numbers) to larger numbers or Curator will return an exception.
An example period
filter demonstrating how to select all daily indices by
timestring found in the index name from last month might look like this:
- filtertype: period period_type: relative source: name range_from: -1 range_to: -1 timestring: '%Y.%m.%d' unit: months
Having range_from
and range_to
both be the same value will mean that only
that whole unit will be selected, in this case, a month.
range_from
and range_to
are required for the relative
pattern type.
Absolute Periods
edit- filtertype: period period_type: absolute source: name timestring: '%Y.%m.%d' unit: months date_from: 2017.01 date_from_format: '%Y.%m' date_to: 2017.01 date_to_format: '%Y.%m'
In addition to relative periods, you can define absolute periods. These
are defined as the period between the date_from
and the
date_to
. For example, if date_from
and date_to
are
both 2017.01
, and unit
is months
, all indices with a
name
, creation_date
, or stats_result
(depending on the value of
source
) within the month of January 2017 will match.
The date_from
is used to establish the beginning of the time period, regardless
of whether date_from_format
is in hours, and the indices you are trying to filter
are in weeks or months. The format and date of date_from
will simply set the
beginning of the time period.
The date_to
, date_to_format
, and unit
work in conjunction to select the
end date. For example, if my date_to
were 2017.04
, and date_to_format
is %Y.%m
to properly parse that date, it would follow that unit
would be
months
.
The period filter is smart enough to calculate months
and years
properly. If unit
is not months
or years
, then your date range will be unit
seconds more than the beginning of the date_from
date, minus 1 second,
according to this table:
units
are calculated as follows:
Unit | Seconds | Note |
---|---|---|
|
|
One second |
|
|
Calculated as 60 seconds |
|
|
Calculated as 60 minutes (60*60) |
|
|
Calculated as 24 hours (24*60*60) |
|
|
Calculated as 7 days (7*24*60*60) |
|
|
Calculated as 30 days (30*24*60*60) |
|
|
Calculated as 365 days (365*24*60*60) |
Specific date ranges can be captured with up to whole second resolution:
- filtertype: period period_type: absolute source: name timestring: '%Y.%m.%d.%H' unit: seconds date_from: 2017-01-01T00:00:00 date_from_format: '%Y-%m-%dT%H:%M:%S' date_to: 2017-01-01T12:34:56 date_to_format: '%Y-%m-%dT%H:%M:%S'
This example will capture indices with an hourly timestamp in their name that fit
between the date_from
and date_to
timestamps.
The identifiers that Curator currently recognizes include:
Unit | Value | Note |
---|---|---|
|
4 digit year |
|
|
4 digit year |
use instead of |
|
2 digit year |
|
|
2 digit month |
|
|
2 digit week of the year |
|
|
2 digit week of the year |
use instead of |
|
2 digit day of the month |
|
|
2 digit hour |
24 hour notation |
|
2 digit minute |
|
|
2 digit second |
|
|
3 digit day of the year |
Dependent settings
edit- range_from
- range_to
- date_from
- date_to
- date_from_format
- date_to_format
-
timestring (required if
source
isname
) -
field (required if
source
isfield_stats
) [Indices only] -
stats_result (only used if
source
isfield_stats
) [Indices only] -
intersect (optional if
source
isfield_stats
) [Indices only]
Optional settings
edit- epoch
-
exclude (default is
False
) - week_starts_on