Ingest processor context

Use a Painless script in an ingest processor to modify documents upon insertion.

Variables

params (Map, read-only)
User-defined parameters passed in as part of the query.
ctx['_index'] (String)
The name of the index.
ctx['_type'] (String)
The type of document within an index.
ctx (Map)
Contains extracted JSON in a Map and List structure for the fields that are part of the document.

Side Effects

ctx['_index']
Modify this to change the destination index for the current document.
ctx['_type']
Modify this to change the type for the current document.
ctx (Map)
Modify the values in the Map/List structure to add, modify, or delete the fields of a document.

Return

void
No expected return value.

API

The standard Painless API is available.

Example

To run this example, first follow the steps in context examples.

The seat data contains:

  • A date in the format YYYY-MM-DD where the second digit of both month and day is optional.
  • A time in the format HH:MM* where the second digit of both hours and minutes is optional. The star (*) represents either the String AM or PM.

The following ingest script processes the date and time Strings and stores the result in a datetime field.

String[] split(String s, char d) {                                   
    int count = 0;

    for (char c : s.toCharArray()) {                                 
        if (c == d) {
            ++count;
        }
    }

    if (count == 0) {
        return new String[] {s};                                     
    }

    String[] r = new String[count + 1];                              
    int i0 = 0, i1 = 0;
    count = 0;

    for (char c : s.toCharArray()) {                                 
        if (c == d) {
            r[count++] = s.substring(i0, i1);
            i0 = i1 + 1;
        }

        ++i1;
    }

    r[count] = s.substring(i0, i1);                                  

    return r;
}

String[] dateSplit = split(ctx.date, (char)"-");                     
String year = dateSplit[0].trim();
String month = dateSplit[1].trim();

if (month.length() == 1) {                                           
    month = "0" + month;
}

String day = dateSplit[2].trim();

if (day.length() == 1) {                                             
    day = "0" + day;
}

boolean pm = ctx.time.substring(ctx.time.length() - 2).equals("PM"); 
String[] timeSplit = split(
        ctx.time.substring(0, ctx.time.length() - 2), (char)":");    
int hours = Integer.parseInt(timeSplit[0].trim());
int minutes = Integer.parseInt(timeSplit[1].trim());

if (pm) {                                                            
    hours += 12;
}

String dts = year + "-" + month + "-" + day + "T" +
        (hours < 10 ? "0" + hours : "" + hours) + ":" +
        (minutes < 10 ? "0" + minutes : "" + minutes) +
        ":00+08:00";                                                 

ZonedDateTime dt = ZonedDateTime.parse(
         dts, DateTimeFormatter.ISO_OFFSET_DATE_TIME);               
ctx.datetime = dt.getLong(ChronoField.INSTANT_SECONDS)*1000L;        

Creates a split function to split a String type value using a char type value as the delimiter. This is useful for handling the necessity of pulling out the individual pieces of the date and time Strings from the original seat data.

The first pass through each char in the String collects how many new Strings the original is split into.

Returns the original String if there are no instances of the delimiting char.

Creates an array type value to collect the split Strings into based on the number of char delimiters found in the first pass.

The second pass through each char in the String collects each split substring into an array type value of Strings.

Collects the last substring into the array type value of Strings.

Uses the split function to separate the date String from the seat data into year, month, and day Strings.

Note
  • The use of a String type value to char type value cast as part of the second argument since character literals do not exist.
  • The use of the ctx ingest processor context variable to retrieve the data from the date field.

Appends the string literal "0" value to a single digit month since the format of the seat data allows for this case.

Appends the string literal "0" value to a single digit day since the format of the seat data allows for this case.

Sets the boolean type variable to true if the time String is a time in the afternoon or evening.

Note
  • The use of the ctx ingest processor context variable to retrieve the data from the time field.

Uses the split function to separate the time String from the seat data into hours and minutes Strings.

Note
  • The use of the substring method to remove the AM or PM portion of the time String.
  • The use of a String type value to char type value cast as part of the second argument since character literals do not exist.
  • The use of the ctx ingest processor context variable to retrieve the data from the date field.

If the time String is an afternoon or evening value adds the integer literal 12 to the existing hours to move to a 24-hour based time.

Builds a new time String that is parsable using existing API methods.

Creates a ZonedDateTime reference type value by using the API method parse to parse the new time String.

Sets the datetime field datetime to the number of milliseconds retrieved from the API method getLong.

Note
  • The use of the ctx ingest processor context variable to set the field datetime. Manipulate each document’s fields with the ctx variable as each document is indexed.

Submit the following request:

PUT /_ingest/pipeline/seats
{
    "description": "update datetime for seats",
    "processors": [
      {
        "script": {
          "source": "String[] split(String s, char d) { int count = 0; for (char c : s.toCharArray()) { if (c == d) { ++count; } } if (count == 0) { return new String[] {s}; } String[] r = new String[count + 1]; int i0 = 0, i1 = 0; count = 0; for (char c : s.toCharArray()) { if (c == d) { r[count++] = s.substring(i0, i1); i0 = i1 + 1; } ++i1; } r[count] = s.substring(i0, i1); return r; } String[] dateSplit = split(ctx.date, (char)\"-\"); String year = dateSplit[0].trim(); String month = dateSplit[1].trim(); if (month.length() == 1) { month = \"0\" + month; } String day = dateSplit[2].trim(); if (day.length() == 1) { day = \"0\" + day; } boolean pm = ctx.time.substring(ctx.time.length() - 2).equals(\"PM\"); String[] timeSplit = split(ctx.time.substring(0, ctx.time.length() - 2), (char)\":\"); int hours = Integer.parseInt(timeSplit[0].trim()); int minutes = Integer.parseInt(timeSplit[1].trim()); if (pm) { hours += 12; } String dts = year + \"-\" + month + \"-\" + day + \"T\" + (hours < 10 ? \"0\" + hours : \"\" + hours) + \":\" + (minutes < 10 ? \"0\" + minutes : \"\" + minutes) + \":00+08:00\"; ZonedDateTime dt = ZonedDateTime.parse(dts, DateTimeFormatter.ISO_OFFSET_DATE_TIME); ctx.datetime = dt.getLong(ChronoField.INSTANT_SECONDS)*1000L;"
        }
      }
    ]
}