In our previous articles, we explored various Spring AI capabilities with simple examples. However, there is one important capability missing from our applications. Our AI assistant cannot remember previous conversations.
This is expected LLM behavior. A model does not automatically remember previous requests. However, we can make our AI application remember a conversation by providing previous messages as context.
This is where conversation memory becomes important.
Why Do We Need Memory?
Without memory, every request is treated independently because the model has no way to understand the previous conversation.
Imagine the following conversation:
User: What is Spring AI?
Assistant: Spring AI is a framework that simplifies AI integration
in Spring applications.
User: Can you explain a bit more?
The second question depends on the previous conversation.
If we send only:
Can you explain a bit more?
the model does not know what the user is referring to. To maintain a proper conversation, we need to provide the previous messages to the model.
How Conversation Memory Works
The model itself does not store our application conversations. Instead, our application stores the conversation messages and provides the relevant previous messages when sending a new request to the model.
Conceptually, the flow looks like this:
User Message -> Load Conversation Memory -> Add Previous Messages -> Send Prompt to LLM -> Receive AI Response -> Update Conversation Memory
We could implement all this logic ourselves.
We would need to:
- Store messages
- Retrieve previous messages
- Identify conversations
- Decide which messages to send
- Update the conversation after each response
Spring AI provides Chat Memory abstractions to simplify this process.
Spring AI Chat Memory
Spring AI provides the ChatMemory abstraction for managing conversation memory.
ChatMemory is responsible for deciding which messages should be maintained as part of the current conversation context. The actual storage and retrieval of messages is handled by a ChatMemoryRepository.
Spring AI provides MessageWindowChatMemory, which maintains a window of recent conversation messages.
@Bean
ChatMemory chatMemory() {
return MessageWindowChatMemory.builder()
.maxMessages(20)
.build();
}
In this example, the conversation memory maintains a window of up to 20 messages.
Because we have not explicitly configured a ChatMemoryRepository, the messages are stored using the default in memory repository.
Conceptually:
MessageWindowChatMemory - Manages the memory window
↓
InMemoryChatMemoryRepository - Stores messages in application memory
Since the messages are stored in application memory, they will be lost when the application restarts. Later, we will learn how to persist conversation memory in a database.
Why Use a Message Window?
Imagine a user chatting with our AI application for three months. The conversation may contain thousands of messages. Sending all previous messages to the model with every request can cause several problems:
- High token usage
- Higher cost
- Increased latency
- Model context window limitations
Instead of sending the complete conversation, we can maintain a limited window of relevant conversation messages. In our example, the conversation memory maintains a window of up to 20 messages.
MessageWindowChatMemory.builder()
.maxMessages(20)
.build();
This helps control the amount of conversation context provided to the model.
Creating a ChatClient with Memory
First, we create the ChatMemory bean as shown above. Next, we provide it to a memory advisor when creating our ChatClient.
@Bean
ChatClient chatClient(
ChatClient.Builder builder,
ChatMemory chatMemory) {
return builder
.defaultAdvisors(
MessageChatMemoryAdvisor
.builder(chatMemory)
.build()
)
.build();
}
Notice that we do not directly interact with ChatMemory when calling the model. Instead, we register a memory advisor with the ChatClient.
What Is a Memory Advisor?
The memory advisor acts as a bridge between ChatClient and ChatMemory.
Before the model is called, the advisor retrieves the relevant conversation messages from ChatMemory and includes them in the request. After the model responds, the conversation memory is updated.
Conceptually:
ChatClient -> Memory Advisor -> ChatMemory -> ChatMemoryRepository
It is important to understand the responsibilities of these components.
ChatMemory
Decides how conversation memory is managed. For example, MessageWindowChatMemory maintains a limited window of messages.
ChatMemoryRepository
Stores and retrieves conversation messages. The repository determines where messages are stored.
For example:
In Memory
Database
Redis
MongoDB
Memory Advisor
Integrates conversation memory into the ChatClient request flow. It retrieves the relevant memory before calling the model and updates the memory as the conversation continues.
Understanding Conversation ID
In real world applications, multiple users and conversations may interact with the same AI application. Therefore, we need a way to separate one conversation from another. This is where the conversation ID becomes important. Each conversation is associated with a unique identifier.
For example:
conversation-1001
conversation-1002
conversation-1003
The memory advisor uses this identifier to retrieve and update the correct conversation memory. We provide the conversation ID through the advisor context.
.advisors(advisorSpec ->
advisorSpec.param(
ChatMemory.CONVERSATION_ID,
conversationId
)
)
It is important to understand that this code does not register a memory advisor. The memory advisor was already registered when we created the ChatClient.
.defaultAdvisors(
MessageChatMemoryAdvisor
.builder(chatMemory)
.build()
)
The following code only provides a parameter to the advisor chain.
.advisors(advisorSpec ->
advisorSpec.param(
ChatMemory.CONVERSATION_ID,
conversationId
)
)
The registered memory advisor recognizes ChatMemory.CONVERSATION_ID and uses it to identify the correct conversation memory.
Conceptually:
Advisor Context -> CONVERSATION_ID = conversation-1001
MessageChatMemoryAdvisor -> Load memory for conversation-1001
ChatClient itself does not use the conversation ID to manage memory. The memory advisor uses it.
Building a Memory-Enabled Endpoint
We have now covered the essential concepts. Let's create a simple REST endpoint that remembers previous messages in a conversation.
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class ChatMemoryController {
private final ChatClient chatClient;
public ChatMemoryController(ChatClient chatClient) {
this.chatClient = chatClient;
}
@PostMapping("/chat")
public String chat(@RequestBody ChatRequest chatRequest) {
return chatClient.prompt()
.user(chatRequest.message())
.advisors(advisorSpec ->
advisorSpec.param(
ChatMemory.CONVERSATION_ID,
chatRequest.conversationId()
)
)
.call()
.content();
}
public record ChatRequest(
String conversationId,
String message
) {
}
}
Notice that the conversation ID is supplied with every request. Let's send the first request.
{
"conversationId": "conversation-1001",
"message": "What is Spring AI?"
}
The model may respond:
Spring AI is a Spring framework that simplifies the integration
of AI models into Java applications.
Now let's send another request using the same conversation ID.
{
"conversationId": "conversation-1001",
"message": "Can you explain a bit more?"
}
Because we use the same conversation ID, the memory advisor retrieves the previous conversation context. The model understands that the user is still asking about Spring AI.
Now consider another request.
{
"conversationId": "conversation-2001",
"message": "Can you explain a bit more?"
}
This is a different conversation. There is no previous conversation context associated with conversation-2001. Therefore, the model may not understand what the user is referring to. This demonstrates how conversation IDs separate memory between conversations.
Chat Memory vs Chat History
It is important to understand that chat memory and chat history are not necessarily the same thing.
Chat memory contains messages that are useful for providing context to the model.
Chat history represents the complete conversation record stored by the application.
Imagine a conversation containing 1,000 messages. Our application may store all 1,000 messages as chat history. However, we may provide only a limited number of messages to the model as conversation memory.
Complete Chat History - 1,000 messages stored
Conversation Memory - Relevant recent messages
LLM Context - Only memory is provided
Chat memory is designed to help the model maintain conversation context. If our application needs a complete permanent conversation history for users, auditing, or reporting, we should store that separately using our application's persistence layer.
Summary
Large Language Models do not automatically remember previous requests. To maintain a conversation, our application needs to provide relevant previous messages as context.
In this article, we explored Spring AI Chat Memory and learned how MessageWindowChatMemory manages a limited conversation memory window.
Comments
Post a Comment