This article will show you how to build an AI agent for HR using GPT-OSS and Elastic Agent Builder. The agent can answer your questions without sending data to OpenAI, Anthropic, or any external service.

We’ll use LM Studio to serve GPT-OSS locally and connect it to Elastic Agent Builder.

By the end of this article, you’ll have a custom AI agent that can answer natural language questions about your employee data while maintaining full control over your information and model.

Prerequisites

For this article, you need:

• Elastic Cloud hosted 9.2, serverless or local deployment

• Machine with 32GB RAM recommended (minimum 16GB for GPT-OSS 20B)

• LM Studio installed

• Docker Desktop Installed

Why use GPT-OSS?

With a local LLM you have the control to deploy it in your own infrastructure and fine-tune it to fit your own needs. All this while maintaining control over the data that you share with the model, and of course, you don’t have to pay a license fee to an external provider.

OpenAI released GPT-OSS on August 5, 2025, as part of their commitment to the open model ecosystem.

The 20B parameter model offers:

• Tool use capabilities

• Efficient inference

• OpenAI SDK compatible

• Compatible with agentic workflows

Benchmark comparison:

Solution architecture

The architecture runs entirely on your local machine. Elastic (running in Docker) communicates directly with your local LLM through LM Studio, and the Elastic Agent Builder uses this connection to create custom AI agents that can query your employee data.

For more details, refer to this documentation.

Building an AI agent for HR: Steps

We’ll divide the implementation into 5 steps:

1. Configure LM studio with a local model

2. Deploy Local Elastic with Docker

3. Create the OpenAI connector in Elastic

4. Upload employee data to Elasticsearch

5. Build and test your AI Agent

Step 1: Configure LM Studio with GPT-OSS 20B

LM Studio is a user-friendly application that allows you to run large language models locally on your computer. It provides an OpenAI-compatible API server, making it easy to integrate with tools like Elastic without a complex setup process. For more details, refer to the LM Studio Docs.

First, download and install LM Studio from the official website. Once installed, open the application.

In the LM Studio interface:

1. Go to the search tab and search for “GPT-OSS”

2. Select the openai/gpt-oss-20b from OpenAI

3. Click download

The size of this model should be approximately 12.10GB. The download may take a few minutes, depending on your internet connection.

Once the model is downloaded:

1. Go to the local server tab

2. Select the openai/gpt-oss-20b

3. Use the default port 1234

4. On the right panel, go to Load and set the Context Length to 40K or higher

5. Click start server

You should see this if the server is running.

Step 2: Deploy Local Elastic with Docker

Now we’ll set up Elasticsearch and Kibana locally using Docker. Elastic provides a convenient script that handles the entire setup process. For more details refer to the official documentation.

Run the start-local script

Execute the following command in your terminal:

This script will:

• Download and configure Elasticsearch and Kibana

• Start both services using Docker Compose

• Automatically activate a 30-day Platinum trial license

Expected output

Just wait for the following message and save the password and API key shown; you’ll need them to access Kibana:

Access Kibana

Open your browser and navigate to:

Log in using the credentials obtained in the terminal output.

Enable Agent Builder

Once logged in to Kibana, navigate to Management > AI > Agent Builder and activate the Agent Builder.

Step 3: Create the OpenAI connector in Elastic

Now we’ll configure Elastic to use your local LLM.

Access Connectors

1. In Kibana

2. Go to Project Settings > Management

3. Under Alerts and Insights, select Connectors

4. Click Create Connector

Configure the connector

Select OpenAI from the list of connectors. LM Studio uses the OpenAI SDK, making it compatible.

Fill in the fields with these values:

• Connector name: LM Studio - GPT-OSS 20B

• Select an OpenAI provider: Other (OpenAI Compatible Service)

• URL: http://host.docker.internal:1234/v1/chat/completions

• Default model: openai/gpt-oss-20b

• API Key: testkey-123 (any text works, because LM Studio Server doesn't require authentication.)

To finish the configuration, click Save & test.

Important: Toggle ON the “Enable native function calling”; this is required for the Agent Builder to work properly. If you don’t enable this, you’ll get a No tool calls found in the response error.

Test the connection

Elastic should automatically test the connection. If everything is configured correctly, you’ll see a success message like this:

Response:

Step 4: Upload employee data to Elasticsearch

Now we’ll upload the HR employee dataset to demonstrate how the agent works with sensitive data. I generated a fictional dataset with this structure.

Dataset structure

Create the index with mappings

First, create the index with proper mappings. Note that we’re using semantic_text fields for some key fields; this enables semantic search capabilities for our index.

Index with Bulk API

Copy and paste the dataset into your Dev Tools in Kibana and execute it:

Verify the data

Run a query to verify:

Step 5: Build and test your AI agent

With everything configured, it’s time to build a custom AI agent using Elastic Agent Builder. For more details refer to the Elastic documentation.

Add the connector

Before we can create our new agent, we have to set our Agent builder to use our custom connector called LM Studio - GPT-OSS 20B because the default one is the Elastic Managed LLM. For that, we need to go to Project Setting > Management > GenAI Settings; now we select the one we created and click Save.

Access Agent Builder

1. Go to Agents

2. Click on Create a new agent

Configure the agent

To create a new agent, the required fields are the Agent ID, Display Name, and Display Instructions.

But there are more customization options, like the Custom Instructions that guide how your agent is going to behave and interact with your tools, similar to a system prompt, but for our custom agent. Labels help organize your agents, avatar color, and avatar symbol.

The ones that I chose for our agent based on the dataset are: Agent ID: hr_assistant

Custom instructions:

Labels: Human Resources and GPT-OSS

Display name: HR Analytics Assistant

Display description:

With all the data in there, we can click on Save our new agent.

Test the agent

Now you can ask natural language questions about your employee data, and GPT-OSS 20B will understand the intent and generate an appropriate response.

Prompt:

Answer:

The Agent process was:

1. Understand your question using the GPT-OSS connector

2. Generate the appropriate Elasticsearch query (using the built-in tools or custom ES|QL)

3. Retrieve matching employee records

4. Present results in natural language with proper formatting

Unlike traditional lexical search, the agent powered by GPT-OSS understands intent and context, making it easier to find information without knowing exact field names or query syntax. For more details on the agent's thinking process, refer to this article.

Conclusion

In this article, we built a custom AI agent using Elastic’s Agent Builder to connect to the OpenAI GPT-OSS model running locally. By deploying both Elastic and the LLM on your local machine, this architecture allows you to leverage generative AI capabilities while maintaining full control over your data, all without sending information to external services.

We used GPT-OSS 20B as an experiment, but the officially recommended models for Elastic Agent Builder are referenced here. If you need more advanced reasoning capabilities, there's also the 120B parameter variant that performs better for complex scenarios, though it requires a higher-spec machine to run locally. For more details, refer to the official OpenAI documentation.

Grounding is the process by which you connect specific data sources to a model to “ground” it to truthful content instead of only relying on the patterns learned during the model’s training, thus giving more reliable and accurate answers.

Grounding helps reduce model hallucinations, generate responses based on your data sources, and allows you to examine the answers by providing citations for them.

Retrieval Augmented Generation(RAG) is a grounding technique where you use search algorithms to retrieve relevant information from external sources, then you use that information as context for the LLM, and finally the model uses the augmented context together with their original training data to generate an answer.

Flow diagram of how RAG works:

RAG allows you to easily scale by updating or expanding the external data sources the model has access to. It is also a cost-effective alternative to fine-tuning LLMs since you can just add data without extensive customization.

And since RAG can access and utilize up-to-date information, it is ideal for use cases when the latest information is key.

Hallucination example

For this example, we’ll use DeepSeek and ask, “Who is the author who won the Chilean National Literature Prize in 1932?” This is a tricky question since the prize was created in 1942. Let’s see how the model answers:

As you can see, since the AI did not have all the information, it hallucinated and provided a made-up answer. Though the information is real in the sense that both the author and work exist, the other parts of the answer are wrong.

Now, let’s see how the model does when we ground it using RAG. For this, we will upload the Wikipedia page about the Chilean National Prize for Literature:

Now, let’s ask the same question and check the answer:

As you can see, with RAG we got the right answer. It says there was no prize in 1932 and asks for clarification from the user.

Using RAG in Playground

By using Elasticsearch, you can easily scale with only having your cluster capacity as a limit. You can use different data sources and connectors to get access to the data you need. Additionally, you have total ownership of your data since it stays in your infrastructure and is not uploaded to a 3rd party service; if you run a local LLM, your data won’t even leave your network. Finally, you have control over search by designing the queries and how to filter data based on access control (RBAC).

We will use Playground, our low-code platform that allows you to quickly and simply create a RAG application using your Elasticsearch content.

Here’s a step-by-step guide on how to upload your PDFs or other documents into Playground. You can also read more about it here and try Playground here.

Upload the PDF

We’ll index into Kibana the same PDF file we provided to DeepSeek. If you followed the instructions in the article above and created the semantic_text field, you’ll be creating a vector database with its corresponding embeddings, ready to be used.

Ask the question

Ask the following question:

“Who is the author who won the Chilean National Literature Prize in 1932?”

Playground sends this query to Elasticsearch, which in turn, runs a semantic search and localizes the fragments with information that is relevant to the question. Then, these fragments are included as context in the prompt sent to the LLM to ground the answer to the information source we provided.

Finally, Playground generated an answer saying that there was no prize in 1932 and provides citations for the relevant fragments as evidence.

Playground also offers two very useful features to understand the RAG system underlying components:

Query

You can see the query Elasticsearch is running to retrieve the relevant documents, and you can enable/disable fields based on your needs.

View code

If you can deploy your RAG application, Playground got you covered. Under the View Code tab, you can see the code used under the hood to create the entire RAG workflow. You can choose between two Python alternatives: Elasticsearch Client with OpenAI, or a Langchain based implementation.

If you want to customize the experience and deploy the code elsewhere, you can use this snippet as a starting point.

Conclusion

Grounding is a process that connects LLMs to external data sources so they can go beyond their training to provide more accurate and trustworthy answers. Retrieval Augmented Generation (RAG) is a grounding method that is scalable, cost-effective, and ensures access to up-to-date information.

Tools like Playground simplify RAG implementation by enabling large-scale indexing, customized searches, and responses with citations, which allow you to easily verify an answer and make sure you’re getting accurate and trustworthy results.

If you want to read more in-depth articles about RAG features, you can start with this one to get a more technical definition of RAG. You can also check Rag vs. fine-tuning: When RAG is the best decision, How to leverage document security using RAG and RAG systems in production.