In the previous article, we built our first Spring AI application and learned a few fundamental concepts. While sending a simple question to an AI model is useful, real-world applications require much more control over how the model behaves.
- A customer support assistant should answer politely and professionally.
- A chess assistant should focus only on chess-related topics.
- An HR assistant should answer based on company policies.
This is where prompt management becomes important.
What Is a Prompt?
A prompt represents the input sent to a Large Language Model (LLM).
In modern LLM applications, a prompt is not always a simple text message. It can contain a sequence of messages, where each message has a specific role in guiding the model.
In Spring AI, a Prompt can contain a collection of messages and optional model configuration.
Understanding Message Types
Modern LLM conversations are built using messages. Each message has a role that helps the model understand the purpose of the content. Spring AI represents these messages using different message types.
System Message
A system message defines the behavior, role, or persona of the model. It tells the model who it is, what it should do, and what constraints it should follow.
For example:
You are a senior Java architect.
Answer only Java and software architecture related questions.
Provide concise and practical answers.
System instructions usually remain stable throughout a conversation.
User Message
A user message represents the request coming from the application user.
For example:
What is a Java Record?
In most applications, this is the input received from the frontend, REST API, or another external system.
Assistant Message
An assistant message represents a response generated by the AI model. Previous assistant messages can be included in a prompt to provide conversation context.
For example, consider the following conversation:
User: What is a Record?
Assistant: A Record is an immutable data carrier.
User: How is it different from a POJO?
The final question depends on the previous conversation. Without the previous messages, the model may not know what "it" refers to. We can explicitly provide these messages in Spring AI.
import org.springframework.ai.chat.messages.AssistantMessage;
import org.springframework.ai.chat.messages.UserMessage;
String response = chatClient.prompt()
.system("You are a Java tutor.")
.messages(
new UserMessage("What is a Record?"),
new AssistantMessage(
"A Record is an immutable data carrier."
)
)
.user("How is it different from a POJO?")
.call()
.content();
Here, we manually provide a previous user question and assistant response. The model receives these messages as conversation context before processing the latest user question.
In this example, we manually construct the conversation history. Later, when we explore Spring AI Memory, we will learn how to manage conversation context automatically.
Tool Response Message
A tool response message represents the result returned from a tool execution. Tools allow an AI model to interact with external systems, APIs, databases, or application logic.
For example, a model may call a weather tool to retrieve the current weather.
User -> LLM -> Weather Tool -> Tool Result -> LLM -> Final Response
In most Spring AI applications, tool calling messages and tool responses are managed automatically by the framework. We will explore tool calling in detail in a future article.
Why Prompt Management Matters
We can easily construct a prompt like this:
String prompt =
"You are a helpful assistant. " +
"Answer politely. " +
"Use less than 100 words. " +
"Question: " + question;
While this works for simple applications, it quickly becomes difficult to maintain as the application grows.
Common problems include:
- Hard-coded prompts
- Duplicated instructions
- Difficult testing
- Difficult prompt updates and versioning
Prompt management helps us organize prompts and separate AI instructions from application logic.
Prompt Templates
Instead of manually concatenating strings, we can use prompt templates. Consider the following template:
Summarize the following text in {style} style:
{text}
The values of style and text can be provided dynamically. Spring AI provides the PromptTemplate class for this purpose.
PromptTemplate template = new PromptTemplate("""
Summarize the following text
in {style} style:
{text}
""");
Map<String, Object> variables = Map.of(
"style", "professional",
"text", "Spring AI simplifies AI integration."
);
Prompt prompt = template.create(variables);
The template variables are replaced when the Prompt is created. This is much cleaner than manually concatenating strings. However, we can improve this further by externalizing our prompts.
Externalizing Prompts
As prompts become larger, keeping them inside Java classes can make the code difficult to maintain. Instead, we can store prompts as external resource files. Spring AI supports String Template files using the .st extension.
Let's create the following files:
src/main/resources/prompts
├── movie-system.st
└── movie-user.st
movie-system.st
You are a movie recommendation assistant.
Recommend exactly {count} movies.
Provide concise and helpful recommendations.
movie-user.st
Recommend movies from the {genre} genre.
Our AI instructions are now separated from the Java application code. We can load these files as Spring resources.
@Value("classpath:/prompts/movie-system.st")
private Resource movieSystem;
@Value("classpath:/prompts/movie-user.st")
private Resource movieUser;
These resources can then be passed directly to ChatClient.
String response = chatClient.prompt()
.system(system -> system.text(movieSystem)
.param("count", count))
.user(user -> user.text(movieUser)
.param("genre", genre))
.call()
.content();
The count and genre template variables are dynamically replaced before the prompt is sent to the model.
Complete Example
Let's combine the concepts we discussed into a complete example. Our controller will demonstrate three prompt management approaches.
Here is the complete controller.
package com.slmanju.springai.controller;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.chat.prompt.PromptTemplate;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.web.bind.annotation.*;
import java.util.Map;
@RestController
public class ChatPromptController {
private final ChatClient chatClient;
@Value("classpath:/prompts/movie-system.st")
private Resource movieSystem;
@Value("classpath:/prompts/movie-user.st")
private Resource movieUser;
public ChatPromptController(ChatClient chatClient) {
this.chatClient = chatClient;
}
@PostMapping("/chat")
public ChatResponse chat(@RequestBody ChatRequest request) {
String response = chatClient.prompt()
.system("You are a senior Java architect.")
.user(request.message())
.call()
.content();
return new ChatResponse(
request.message(),
response
);
}
@PostMapping("/summarize")
public ChatResponse summarize(
@RequestBody ChatRequest request) {
PromptTemplate promptTemplate = new PromptTemplate("""
Summarize the following text
in {style} style:
{text}
""");
Map<String, Object> variables = Map.of(
"style", "bullet point",
"text", request.message()
);
Prompt prompt = promptTemplate.create(variables);
String response = chatClient.prompt(prompt)
.call()
.content();
return new ChatResponse(
request.message(),
response
);
}
@GetMapping("/movies")
public String movies(
@RequestParam String genre,
@RequestParam(defaultValue = "5") Integer count) {
return chatClient.prompt()
.system(system -> system
.text(movieSystem)
.param("count", count))
.user(user -> user
.text(movieUser)
.param("genre", genre))
.call()
.content();
}
public record ChatRequest(String message) {
}
public record ChatResponse(
String request,
String response) {
}
}
- The
/chatendpoint demonstrates how system and user messages can be used to define the model's behavior and provide user input. - The
/summarizeendpoint demonstrates howPromptTemplatecan dynamically construct a prompt using template variables. - The
/moviesendpoint demonstrates how system and user prompts can be externalized into resource files.
Summary
Prompt design is one of the most important skills in AI development. Well structured prompts can significantly improve the quality and consistency of model responses.
In this article, we explored how Spring AI represents prompts as a collection of messages and learned about different message roles.
In the next article, we will explore structured output and learn how to map AI responses directly into Java records and POJOs.
Comments
Post a Comment