Spring AI: Tool Calling with Spring AI

Greetings!

As we slowly move through our journey of learning Spring AI, there is something important we need to understand. An LLM does not know about your application environment. What does that mean? Think of a simple question like this:

"What is the current date and time?"

LLMs are trained using data from the past. They do not automatically have access to the current date and time, your database, your APIs, or your application's data.

By the end of this article, our AI assistant will be able to use Java methods to access such information and perform actions.

Why Tool Calling Exists

LLMs are excellent at:

Explaining
Summarizing
Reasoning
Generating text
And much more

However, they cannot directly:

Access real-time information
Access databases
Call application APIs
Execute business actions

This is why we need tools. Tools allow the model to interact with the outside world through our application.

Please remember that not all AI models support tool calling.

What Is a Tool?

A tool is simply a function that we make available to the model.

However, the LLM does not directly execute our Java method. Instead, the model decides whether a tool is required and requests a tool call with the necessary arguments.

Spring AI executes the corresponding Java method, sends the tool result back to the model, and continues the conversation until the model produces a final response.

Creating Our First Tool

Let's provide our LLM with the capability to answer questions about the current date and time.

java

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.context.i18n.LocaleContextHolder;
import org.springframework.stereotype.Component;

import java.time.LocalDateTime;

@Component
public class AiTools {

    @Tool(description = "Get the current date and time in the user's timezone")
    public String getCurrentDateTime() {
        return LocalDateTime.now()
                .atZone(LocaleContextHolder.getTimeZone().toZoneId())
                .toString();
    }

}

When a tool is registered, Spring AI includes its definition in the request sent to the model. The model uses the tool name, description, and parameter information to determine whether the tool is relevant to the user's request.

Therefore, it is important to provide a clear and meaningful tool description. As you can see, this is just another Spring bean. We are free to perform application logic inside the method, such as accessing a database, calling an API, or invoking another service.

Registering Tools

We can register tools at two points:

When initializing the ChatClient
When invoking the ChatClient

First, let's register our date and time tool as a default tool.

java

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AiConfiguration {

    @Bean
    public ChatClient chatClient(
            ChatClient.Builder builder,
            AiTools aiTools) {

        return builder
                .defaultTools(aiTools)
                .build();
    }

}

Here, we initialize the ChatClient with a default tool. This tool will be available to requests made using this ChatClient.

Now, let's create another tool and provide it at invocation time.

Creating a Tool with Parameters

Most real world tools require parameters. For example, an application may need to search for products in a database.

Let's create a simple product search tool.

java

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.stereotype.Component;

import java.util.List;

@Component
public class ProductTools {

    private final List<Product> products = List.of(
            new Product(1L, "Laptop", "Electronics", 1200.00),
            new Product(2L, "Headphones", "Electronics", 150.00),
            new Product(3L, "Office Chair", "Furniture", 300.00)
    );

    @Tool(description = "Find products by category")
    public List<Product> findProducts(
            @ToolParam(description = "Product category")
            String category) {

        return products.stream()
                .filter(product ->
                        product.category().equalsIgnoreCase(category))
                .toList();
    }

    public record Product(
            Long id,
            String name,
            String category,
            double price
    ) {}

}

Here, we are exploring tool parameters. In real world applications, we will usually encounter tools that accept parameters rather than tools that simply execute an action without any input.

For example, consider the following user request:

"Find electronics products."

The model can understand the request, select the findProducts tool, and determine that the category parameter should be Electronics.

Conceptually, the model requests a tool call with arguments similar to:

json

{
  "category": "Electronics"
}

Spring AI then executes the corresponding Java method.

Providing Tools at Invocation Time

Let's provide our ProductTools when invoking the ChatClient.

java

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ToolsController {

    private final ChatClient chatClient;
    private final ProductTools productTools;

    public ToolsController(
            ChatClient chatClient,
            ProductTools productTools) {

        this.chatClient = chatClient;
        this.productTools = productTools;
    }

    @PostMapping("/chat")
    public String chat(@RequestBody ChatRequest request) {
        return chatClient.prompt()
                .user(request.message())
                .tools(productTools)
                .call()
                .content();
    }

    public record ChatRequest(String message) {}

}

It is as simple as that.

We do not need to manually inspect the user's request, select a Java method, extract the parameters, or execute the method ourselves.

The model decides whether a tool is required and provides the necessary arguments. Spring AI handles the tool calling flow and executes the requested tool.

Summary

Tools are the bridge between an LLM and our application. They allow the model to work with real time information, databases, APIs, and business operations.

In the next articles, we will see how these capabilities become an important building block for AI agents.

Manju