Transform limitationsedit
This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
The following limitations and known problems apply to the 7.2 release of the Elastic data frame feature:
Data frame data type limitationedit
Data frames do not (yet) support fields containing arrays – in the UI or the API. If you try to create one, the UI will fail to show the source index table.
Cross-cluster search limitationedit
Cross-cluster search is not supported in 7.2 for transforms.
Kibana only displays up to 100 transformsedit
The Kibana Machine Learning > Data Frames page lists up to 100
transforms. You can work-around this limitation by calling the
GET transforms API
with the size
parameter.
Date histogram limitationedit
If you use a date
histogram in the group_by
object in the create or preview transform
APIs, the defined interval and time format must have the same time fidelity.
Otherwise, it might cause issues in the data frame.
For example, if you set the calendar_interval
of the date histogram to one minute
(1m
), then make sure that the format
is yyyy-MM-dd HH:mm
instead of
yyyy-MM-dd HH:00
.
Date format limitation in transform destination indexedit
When you create a transform that uses a date_histogram
as a group-by
and set the interval
to 1y
, the date could be interpreted incorrectly
in the generated date field of the destination index. The reason is that the yyyy
value can be identified incorrectly as epoch_millis
. As a workaround, using the
API, you may define a custom destination index data format mapping prior to starting
the transform. For example:
"mappings" : { "properties" : { "custom_date" : { "type" : "date", "format": "yyyy" } } }
Aggregation responses may be incompatible with destination index mappingsedit
Data frames use composite aggregations to transform data. In some cases, composite aggregations may return responses which are not compatible with the mappings set for the destination index. For example "NaN", "Infinity" or possibly a numeric overflow. Where possible, a null response has been substituted. Please, check Elasticsearch logs if you think this may have occurred. As a workaround, using the API, you may define custom destination index mappings prior to starting the transform.