Loading

Build a custom agent with the Elastic Agent Builder APIs

Learn how to work with Elastic Agent Builder programmatically using the Kibana APIs. This tutorial walks you through creating custom tools and agents that can help users search and analyze data.

This tutorial assumes you have basic knowledge of:

  • Elasticsearch indices and data structures
  • REST APIs and JSON
  • Basic ES|QL query syntax

By the end of this tutorial, you can:

  • Create and run custom ES|QL tools tailored to specific use cases
  • Build custom agents with specific instructions and tool sets
  • Chat with agents programmatically using the converse API
  • Manage conversations and clean up resources

To build a custom agent with Elastic Agent Builder APIs, you need the following:

  • Access to the feature in Kibana. Refer to Get started with Elastic Agent Builder
  • Permission to create indices and use Elastic Agent Builder
    • A Kibana API key for authentication if you're using curl
    Note

    Set the required environment variables to simplify running curl commands:

    export KIBANA_URL="your-kibana-url"
    export ELASTICSEARCH_URL="your-elasticsearch-url"
    export API_KEY="your-api-key"
    		

Before working with agents and tools, create a sample dataset. This tutorial uses a collection of books to demonstrate how custom tools can efficiently query specific data.

Create an index with book data:

				PUT /kibana_sample_data_agents
					{
  "mappings": {
    "properties": {
      "name": { "type": "text" },
      "author": { "type": "text" },
      "release_date": { "type": "date" },
      "page_count": { "type": "integer" }
    }
  }
}
		
curl -X PUT "${ELASTICSEARCH_URL}/kibana_sample_data_agents" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "Content-Type: application/json" \
     -d '{
       "mappings": {
         "properties": {
           "name": { "type": "text" },
           "author": { "type": "text" },
           "release_date": { "type": "date" },
           "page_count": { "type": "integer" }
         }
       }
     }'
		
Tip

If you're using Spaces, you need to prefix /api/agent_builder with /s/<space_name>. Refer to Working with Spaces.

Add sample book documents to the index:

				POST /_bulk
					{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Snow Crash", "author": "Neal Stephenson", "release_date": "1992-06-01", "page_count": 470}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Revelation Space", "author": "Alastair Reynolds", "release_date": "2000-03-15", "page_count": 585}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "1984", "author": "George Orwell", "release_date": "1985-06-01", "page_count": 328}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Fahrenheit 451", "author": "Ray Bradbury", "release_date": "1953-10-15", "page_count": 227}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Brave New World", "author": "Aldous Huxley", "release_date": "1932-06-01", "page_count": 268}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "The Handmaids Tale", "author": "Margaret Atwood", "release_date": "1985-06-01", "page_count": 311}
		
curl -X POST "${ELASTICSEARCH_URL}/_bulk" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "Content-Type: application/x-ndjson" \
     --data-binary @- << 'EOF'
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Snow Crash", "author": "Neal Stephenson", "release_date": "1992-06-01", "page_count": 470}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Revelation Space", "author": "Alastair Reynolds", "release_date": "2000-03-15", "page_count": 585}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "1984", "author": "George Orwell", "release_date": "1985-06-01", "page_count": 328}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Fahrenheit 451", "author": "Ray Bradbury", "release_date": "1953-10-15", "page_count": 227}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "Brave New World", "author": "Aldous Huxley", "release_date": "1932-06-01", "page_count": 268}
{ "index" : { "_index" : "kibana_sample_data_agents" } }
{"name": "The Handmaids Tale", "author": "Margaret Atwood", "release_date": "1985-06-01", "page_count": 311}
EOF
		

You now have a kibana_sample_data_agents index with six books, ready for querying with custom tools.

Start by chatting with the default agent to understand how Elastic Agent Builder works. The default agent has access to built-in tools that can query your data.

Send a message to the default agent:

				POST kbn://api/agent_builder/converse
					{
  "input": "What books are in the kibana_sample_data_agents index?"
}
		
curl -X POST "${KIBANA_URL}/api/agent_builder/converse" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "input": "What books are in the kibana_sample_data_agents index?"
     }'
		

The agent responds with information about the books in your sample data. Notice the token_usage field in the response. To learn how to track token consumption, refer to Monitor token usage. You'll compare this with your custom agent later to see how specialized tools can reduce token consumption.

Tools are reusable functions that agents use to perform specific tasks. Elastic Agent Builder includes built-in tools for common operations like searching indices and generating queries. You can also create custom tools.

List all available tools:

				GET kbn://api/agent_builder/tools
		
curl -X GET "${KIBANA_URL}/api/agent_builder/tools" \
     -H "Authorization: ApiKey ${API_KEY}"
		

The response includes all available tools, including built-in platform tools like platform.core.search, platform.core.generate_esql, and others.

Use the built-in ES|QL generator tool to create a query for your sample data. This tool generates optimized ES|QL queries based on natural language descriptions.

Run the ES|QL generator tool:

				POST kbn://api/agent_builder/tools/_execute
					{
  "tool_id": "platform.core.generate_esql",
  "tool_params": {
    "query": "Build an ES|QL query to get the book with the most pages",
    "index": "kibana_sample_data_agents"
  }
}
		
  1. ID of the built-in ES|QL generator tool
  2. Natural language description of the desired query
  3. Index to query
curl -X POST "${KIBANA_URL}/api/agent_builder/tools/_execute" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "tool_id": "platform.core.generate_esql",
       "tool_params": {
         "query": "Build an ES|QL query to get the book with the most pages",
         "index": "kibana_sample_data_agents"
       }
     }'
		
  1. ID of the built-in ES|QL generator tool
  2. Natural language description of the desired query
  3. Index to query

The tool returns an ES|QL query similar to: FROM kibana_sample_data_agents | SORT page_count DESC | LIMIT 1. You'll use this query in the next step to create a custom tool.

Create a custom ES|QL tool using the query from the previous step. Custom tools encapsulate specific queries or operations, making them reusable and efficient.

Create a tool that finds the book with the most pages:

				POST kbn://api/agent_builder/tools
					{
  "id": "example-books-esql-tool",
  "type": "esql",
  "description": "An ES|QL query tool for getting the book with the most pages",
  "configuration": {
    "query": "FROM kibana_sample_data_agents | SORT page_count DESC | LIMIT 1",
    "params": {}
  }
}
		
  1. Unique identifier for the tool
  2. Tool type - esql for ES|QL query tools
  3. Description that helps agents understand when to use this tool
  4. The ES|QL query to run
  5. Query parameters (empty for this basic example)
curl -X POST "${KIBANA_URL}/api/agent_builder/tools" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "id": "example-books-esql-tool",
       "type": "esql",
       "description": "An ES|QL query tool for getting the book with the most pages",
       "configuration": {
         "query": "FROM kibana_sample_data_agents | SORT page_count DESC | LIMIT 1",
         "params": {}
       }
     }'
		
  1. Unique identifier for the tool
  2. Tool type - esql for ES|QL query tools
  3. Description that helps agents understand when to use this tool
  4. The ES|QL query to run
  5. Query parameters (empty for this basic example)

The response confirms the tool was created with its full configuration.

Test your new custom tool to verify it works correctly.

Run the custom tool:

				POST kbn://api/agent_builder/tools/_execute
					{
  "tool_id": "example-books-esql-tool",
  "tool_params": {}
}
		
curl -X POST "${KIBANA_URL}/api/agent_builder/tools/_execute" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "tool_id": "example-books-esql-tool",
       "tool_params": {}
     }'
		

The response includes tabular data showing "Revelation Space" by Alastair Reynolds with 585 pages.

Retrieve the details of a specific tool using its ID.

Get the tool you created:

				GET kbn://api/agent_builder/tools/example-books-esql-tool
		
curl -X GET "${KIBANA_URL}/api/agent_builder/tools/example-books-esql-tool" \
     -H "Authorization: ApiKey ${API_KEY}"
		

The response includes the full tool definition with all its configuration details.

Make your tool more flexible by adding parameters. This allows the same tool to handle different queries based on input values.

Update the tool to filter by year and limit results:

				PUT kbn://api/agent_builder/tools/example-books-esql-tool
					{
  "description": "An ES|QL query tool for finding the longest books published before a certain year",
  "configuration": {
    "query": "FROM kibana_sample_data_agents | WHERE DATE_EXTRACT(\"year\", release_date) < ?maxYear | SORT page_count DESC | LIMIT ?limit",
    "params": {
      "maxYear": {
        "type": "integer",
        "description": "Maximum year to filter books (exclusive)"
      },
      "limit": {
        "type": "integer",
        "description": "Maximum number of results to return"
      }
    }
  }
}
		
  1. Query with parameterized placeholders (?maxYear, ?limit)
  2. Integer parameter for filtering by publication year
  3. Integer parameter for limiting results
curl -X PUT "${KIBANA_URL}/api/agent_builder/tools/example-books-esql-tool" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "description": "An ES|QL query tool for finding the longest books published before a certain year",
       "configuration": {
         "query": "FROM kibana_sample_data_agents | WHERE DATE_EXTRACT(\"year\", release_date) < ?maxYear | SORT page_count DESC | LIMIT ?limit",
         "params": {
           "maxYear": {
             "type": "integer",
             "description": "Maximum year to filter books (exclusive)"
           },
           "limit": {
             "type": "integer",
             "description": "Maximum number of results to return"
           }
         }
       }
     }'
		
  1. Query with parameterized placeholders (?maxYear, ?limit)
  2. Integer parameter for filtering by publication year
  3. Integer parameter for limiting results

Run the updated tool with parameters:

				POST kbn://api/agent_builder/tools/_execute
					{
  "tool_id": "example-books-esql-tool",
  "tool_params": {
    "maxYear": 1960,
    "limit": 2
  }
}
		
  1. Find books published before 1960
  2. Return only the top 2 results
curl -X POST "${KIBANA_URL}/api/agent_builder/tools/_execute" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "tool_id": "example-books-esql-tool",
       "tool_params": {
         "maxYear": 1960,
         "limit": 2
       }
     }'
		
  1. Find books published before 1960
  2. Return only the top 2 results

The response shows "Brave New World" (268 pages, 1932) and "Fahrenheit 451" (227 pages, 1953), the two longest books published before 1960.

Agents combine instructions with specific tools to handle user conversations effectively. Create an agent specialized for searching the books collection.

List all existing agents to see what's available:

				GET kbn://api/agent_builder/agents
		
curl -X GET "${KIBANA_URL}/api/agent_builder/agents" \
     -H "Authorization: ApiKey ${API_KEY}"
		

Create a specialized agent for book searches:

				POST kbn://api/agent_builder/agents
					{
  "id": "books-search-agent",
  "name": "Books Search Helper",
  "description": "Hi! I can help you search and analyze the books in our sample data collection.",
  "labels": ["books", "sample-data", "search"],
  "avatar_color": "#BFDBFF",
  "avatar_symbol": "📚",
  "configuration": {
    "instructions": "You are a helpful agent that assists users in searching and analyzing book data from the kibana_sample_data_agents index. Help users find books by author, title, or analyze reading patterns.",
    "tools": [
      {
        "tool_ids": [
          "example-books-esql-tool",
          "platform.core.search",
          "platform.core.list_indices",
          "platform.core.get_index_mapping",
          "platform.core.get_document_by_id"
        ]
      }
    ]
  }
}
		
  1. Unique identifier for the agent
  2. Display name shown in the UI
  3. Greeting message users view when starting a conversation
  4. Labels for organizing and filtering agents
  5. Avatar background color (hex code)
  6. Avatar symbol or emoji
  7. System instructions that guide the agent's behavior
  8. Tools the agent can use - includes your custom tool and built-in tools
curl -X POST "${KIBANA_URL}/api/agent_builder/agents" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "id": "books-search-agent",
       "name": "Books Search Helper",
       "description": "Hi! I can help you search and analyze the books in our sample data collection.",
       "labels": ["books", "sample-data", "search"],
       "avatar_color": "#BFDBFF",
       "avatar_symbol": "📚",
       "configuration": {
         "instructions": "You are a helpful agent that assists users in searching and analyzing book data from the kibana_sample_data_agents index. Help users find books by author, title, or analyze reading patterns.",
         "tools": [
           {
             "tool_ids": [
               "example-books-esql-tool",
               "platform.core.search",
               "platform.core.list_indices",
               "platform.core.get_index_mapping",
               "platform.core.get_document_by_id"
             ]
           }
         ]
       }
     }'
		
  1. Unique identifier for the agent
  2. Display name shown in the UI
  3. Greeting message users view when starting a conversation
  4. Labels for organizing and filtering agents
  5. Avatar background color (hex code)
  6. Avatar symbol or emoji
  7. System instructions that guide the agent's behavior
  8. Tools the agent can use - includes your custom tool and built-in tools

The response confirms the agent was created with its full configuration.

Retrieve the details of a specific agent using its ID.

Get the agent you created:

				GET kbn://api/agent_builder/agents/books-search-agent
		
curl -X GET "${KIBANA_URL}/api/agent_builder/agents/books-search-agent" \
     -H "Authorization: ApiKey ${API_KEY}"
		

The response includes the full agent definition with all its configuration details.

Update your agent's configuration, such as changing its description or labels.

Update the agent's description:

				PUT kbn://api/agent_builder/agents/books-search-agent
					{
  "name": "Books Search Helper",
  "description": "Updated - Search and analyze our sample books collection with ease!",
  "labels": ["books", "sample-data", "search", "updated"]
}
		
curl -X PUT "${KIBANA_URL}/api/agent_builder/agents/books-search-agent" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "name": "Books Search Helper",
       "description": "Updated - Search and analyze our sample books collection with ease!",
       "labels": ["books", "sample-data", "search", "updated"]
     }'
		

The response confirms the agent was updated with the new configuration.

Test your new agent by asking it questions about the book collection. The agent uses your custom tool and built-in tools to answer queries.

Start a conversation with the custom agent:

				POST kbn://api/agent_builder/converse
					{
  "input": "What books do we have in our collection?",
  "agent_id": "books-search-agent"
}
		
  1. Specify your custom agent ID
curl -X POST "${KIBANA_URL}/api/agent_builder/converse" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "input": "What books do we have in our collection?",
       "agent_id": "books-search-agent"
     }'
		
  1. Specify your custom agent ID

The agent responds with information about your book collection. Note the conversation_id in the response - you'll use this to continue the conversation.

Continue the conversation using the custom tool:

				POST kbn://api/agent_builder/converse
					{
  "input": "Can you find the longest book published before 1960?",
  "agent_id": "books-search-agent",
  "conversation_id": "<CONVERSATION_ID>"
}
		
  1. Use the conversation ID from the previous response to maintain context
curl -X POST "${KIBANA_URL}/api/agent_builder/converse" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true" \
     -H "Content-Type: application/json" \
     -d '{
       "input": "Can you find the longest book published before 1960?",
       "agent_id": "books-search-agent",
       "conversation_id": "<CONVERSATION_ID>"
     }'
		
  1. Use the conversation ID from the previous response to maintain context

The agent uses your custom example-books-esql-tool to efficiently answer the query in a single step. Compare the token_usage in this response with Step 1. Custom tools optimized for specific use cases typically consume fewer tokens than general-purpose agents. To learn more about token consumption, refer to Monitor token usage.

Tip

For real-time chat responses, use the streaming converse API instead.

View and manage your conversation history with agents.

List all conversations:

				GET kbn://api/agent_builder/conversations
		
curl -X GET "${KIBANA_URL}/api/agent_builder/conversations" \
     -H "Authorization: ApiKey ${API_KEY}"
		

Get the full history of a specific conversation:

				GET kbn://api/agent_builder/conversations/<CONVERSATION_ID>
		
  1. Replace with an actual conversation ID from the previous step
curl -X GET "${KIBANA_URL}/api/agent_builder/conversations/<CONVERSATION_ID>" \
     -H "Authorization: ApiKey ${API_KEY}"
		
  1. Replace with an actual conversation ID from the previous step

The response includes the complete conversation history with all messages, tool calls, and responses.

Remove the resources created in this tutorial when you no longer need them.

Delete the conversation:

				DELETE kbn://api/agent_builder/conversations/<CONVERSATION_ID>
		
curl -X DELETE "${KIBANA_URL}/api/agent_builder/conversations/<CONVERSATION_ID>" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true"
		

Delete the custom agent:

				DELETE kbn://api/agent_builder/agents/books-search-agent
		
curl -X DELETE "${KIBANA_URL}/api/agent_builder/agents/books-search-agent" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true"
		

Delete the custom tool:

				DELETE kbn://api/agent_builder/tools/example-books-esql-tool
		
curl -X DELETE "${KIBANA_URL}/api/agent_builder/tools/example-books-esql-tool" \
     -H "Authorization: ApiKey ${API_KEY}" \
     -H "kbn-xsrf: true"
		

Delete the sample index:

				DELETE /kibana_sample_data_agents
		
curl -X DELETE "${ELASTICSEARCH_URL}/kibana_sample_data_agents" \
     -H "Authorization: ApiKey ${API_KEY}"
		

All resources created during this tutorial are now removed.

In this tutorial, you learned how to:

  • Create custom ES|QL tools with and without parameters
  • Run tools directly using the tools API
  • Build custom agents with specific instructions and tool sets
  • Chat with agents programmatically using the converse API
  • Manage conversation history

Custom tools optimized for specific use cases can significantly reduce token consumption and improve response accuracy compared to general-purpose agents with many tools. By tailoring agents and tools to your specific data and workflows, you create more efficient and effective AI-powered experiences.