Body Required

• chunking_settings object

  The chunking configuration object. Applies only to the text_embedding task type. Not applicable to the rerank, completion, or chat_completion task types.

  External documentation
  Hide chunking_settings attributes Show chunking_settings attributes object

  • max_chunk_size number

    The maximum size of a chunk in words. This value cannot be lower than 20 (for sentence strategy) or 10 (for word strategy). This value should not exceed the window size for the associated model.

    Default value is 250.

  • overlap number

    The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    Default value is 100.

  • sentence_overlap number

    The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    Default value is 1.

  • separator_group string

    Only applicable to the recursive strategy and required when using it.

    Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be markdown or plaintext.

    Using this parameter is an alternative to manually specifying a custom separators list.

  • separators array[string]

    Only applicable to the recursive strategy and required when using it.

    A list of strings used as possible split points when chunking text.

    Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.

    After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the max_chunk_size limit, to reduce the total number of chunks generated.

  • strategy string

    The chunking strategy: sentence, word, none or recursive.

    • If strategy is set to recursive, you must also specify:

      • max_chunk_size
      • either separators orseparator_group

    Learn more about different chunking strategies in the linked documentation.

    Default value is sentence.

    External documentation
• service string Required

  The type of service supported for the specified task type. In this case, googlevertexai.

  Value is googlevertexai.

• service_settings object Required

  Settings used to install the inference model. These settings are specific to the googlevertexai service.

  Hide service_settings attributes Show service_settings attributes object

  • provider string

    The name of the Google Model Garden Provider for completion and chat_completion tasks. In order for a Google Model Garden endpoint to be used provider must be defined and be other than google. Modes:

    • Google Model Garden (third-party models): set provider to a supported non-google value and provide url and/or streaming_url.
    • Google Vertex AI: omit provider or set it to google. In this mode, do not set url or streaming_url and Elastic will construct the endpoint url from location, model_id, and project_id parameters.

    Values are google, anthropic, meta, hugging_face, mistral, or ai21.

  • url string

    The URL for non-streaming completion requests to a Google Model Garden provider endpoint. If both url and streaming_url are provided, each is used for its respective mode. If streaming_url is not provided, url is also used for streaming completion and chat_completion. If provider is not provided or set to google (Google Vertex AI), do not set url (or streaming_url). At least one of url or streaming_url must be provided for Google Model Garden endpoint usage. Certain providers require separate URLs for streaming and non-streaming operations (e.g., Anthropic, Mistral, AI21). Others support both operation types through a single URL (e.g., Meta, Hugging Face). Information on constructing the URL for various providers can be found in the Google Model Garden documentation for the model, or on the endpoint’s Sample request page. The request examples also illustrate the proper formatting for the url.

  • streaming_url string

    The URL for streaming completion and chat_completion requests to a Google Model Garden provider endpoint. If both streaming_url and url are provided, each is used for its respective mode. If url is not provided, streaming_url is also used for non-streaming completion requests. If provider is not provided or set to google (Google Vertex AI), do not set streaming_url (or url). At least one of streaming_url or url must be provided for Google Model Garden endpoint usage. Certain providers require separate URLs for streaming and non-streaming operations (e.g., Anthropic, Mistral, AI21). Others support both operation types through a single URL (e.g., Meta, Hugging Face). Information on constructing the URL for various providers can be found in the Google Model Garden documentation for the model, or on the endpoint’s Sample request page. The request examples also illustrate the proper formatting for the streaming_url.

  • location string

    The name of the location to use for the inference task for the Google Vertex AI inference task. For Google Vertex AI, when provider is omitted or google location is mandatory. For Google Model Garden's completion and chat_completion tasks, when provider is a supported non-google value - location is ignored. Refer to the Google documentation for the list of supported locations.

    External documentation
  • model_id string

    The name of the model to use for the inference task. For Google Vertex AI model_id is mandatory. For Google Model Garden's completion and chat_completion tasks, when provider is a supported non-google value - model_id will be used for some providers that require it, otherwise - ignored. Refer to the Google documentation for the list of supported models for Google Vertex AI.

    External documentation
  • project_id string

    The name of the project to use for the Google Vertex AI inference task. For Google Vertex AI project_id is mandatory. For Google Model Garden's completion and chat_completion tasks, when provider is a supported non-google value - project_id is ignored.

  • rate_limit object

    This setting helps to minimize the number of rate limit errors returned from Google Vertex AI. By default, the googlevertexai service sets the number of requests allowed per minute to 30.000.

    Hide rate_limit attribute Show rate_limit attribute object

    • requests_per_minute number

      The number of requests allowed per minute. By default, the number of requests allowed per minute is set by each service as follows:

      • alibabacloud-ai-search service: 1000
      • amazonbedrock service: 240
      • anthropic service: 50
      • azureaistudio service: 240
      • azureopenai service and task type text_embedding: 1440
      • azureopenai service and task types completion or chat_completion: 120
      • cohere service: 10000
      • contextualai service: 1000
      • elastic service and task type chat_completion: 240
      • googleaistudio service: 360
      • googlevertexai service: 30000
      • hugging_face service: 3000
      • jinaai service: 2000
      • llama service: 3000
      • mistral service: 240
      • openai service and task type text_embedding: 3000
      • openai service and task type completion: 500
      • openshift_ai service: 3000
      • voyageai service: 2000
      • watsonxai service: 120
  • service_account_json string Required

    A valid service account in JSON format for the Google Vertex AI API.

  • dimensions number

    For a text_embedding task, the number of dimensions the resulting output embeddings should have. By default, the model's standard output dimension is used. Refer to the Google documentation for more information.

    External documentation
  • max_batch_size number

    Only applicable for the text_embedding task type. Controls the batch size of chunked inference requests sent to Google Vertex AI.

    Setting this parameter lower reduces the risk of exceeding token limits but may result in more API calls. Setting it higher increases throughput but may risk hitting token limits.

    To estimate a safe max_batch_size value, you can use it together with the max_chunk_size parameter using the following formula: max_batch_size ≈ max_chunk_size × 1.3 × 512 ÷ 20000

    Where:

    • 1.3 is an approximate tokens-per-word ratio
    • 512 is the maximum number of chunks that can be generated per document
    • 20000 is the Google Vertex AI token limit per request

    This estimate assumes the worst-case scenario with a document generating the maximum 512 chunks.

    Default value is 250.

• task_settings object

  Settings to configure the inference task. These settings are specific to the task type you specified.

  Hide task_settings attributes Show task_settings attributes object

  • auto_truncate boolean

    For a text_embedding task, truncate inputs longer than the maximum token length automatically.

  • top_n number

    For a rerank task, the number of the top N documents that should be returned.

  • thinking_config object

    For a completion or chat_completion task, allows configuration of the thinking features for the model. Refer to the Google documentation for the allowable configurations for each model type.

    External documentation
    Hide thinking_config attribute Show thinking_config attribute object

    • thinking_budget number

      Indicates the desired thinking budget in tokens.

  • max_tokens number

    For completion and chat_completion tasks, specifies the max_tokens value for requests sent to the Google Model Garden anthropic provider. If provider is not set to anthropic, this field is ignored. If max_tokens is specified - it must be a positive integer. If not specified, the default value of 1024 is used. Anthropic models require max_tokens to be set for each request. Please refer to the Anthropic documentation for more information.

    External documentation