In previous articles, we used Spring AI to generate responses in text format. While generating text is useful, enterprise applications rarely need only free form responses. Applications usually need predictable data that can be processed by application logic. Imagine we need to extract customer details using an LLM.
Extract customer information:
John Smith
john@example.com
+1 555-1234
The LLM may correctly return a response like this:
Name: John Smith
Email: john@example.com
Phone: +1 555-1234
While this looks fine to the human eye, our application cannot reliably parse this kind of response. The model may change the format.
Customer Name: John Smith
Email Address: john@example.com
Contact Number: +1 555-1234
Or it may return a sentence.
The customer's name is John Smith. His email address is
john@example.com and his phone number is +1 555-1234.
All these responses are understandable to a human, but application code needs a well defined structure. This is where structured output becomes useful.
What Is Structured Output?
Instead of working with free form text responses, we ask the model to generate output that follows a defined structure.
Spring AI can map the model response into a Java type, giving our application predictable and type safe data that is easier to process and validate.
For example, instead of parsing text, we want to work with a Java object.
Customer customer;
Then our application can simply access the extracted values.
customer.name();
customer.email();
customer.phone();
Mapping Responses to Java Records
Spring AI provides built in support for mapping model responses to Java types.
Let's define a Java record representing our customer.
public record Customer(
String name,
String email,
String phone
) {
}
Now we can ask the model to extract the customer information and map the response directly to our Customer record.
Customer customer = chatClient.prompt()
.user("""
Extract customer information:
John Smith
john@example.com
+1 555-1234
""")
.call()
.entity(Customer.class);
The entity() method tells Spring AI to convert the model response into the specified Java type.
Spring AI uses the target type to describe the expected output structure to the model and converts the generated response into a Customer object. Instead of manually parsing the response, we can now work with a normal Java object.
customer.name();
customer.email();
customer.phone();
Java records are particularly useful for structured AI responses because they are:
- Immutable
- Concise
- Easy to serialize
- Easy to validate
Mapping Collections
Structured output also supports collections. Imagine we want the model to recommend five books for Java developers.
First, let's define a Book record.
public record Book(
String title,
String author,
String summary
) {
}
We can map the model response directly to a List<Book>.
List<Book> books = chatClient.prompt()
.user("Recommend 5 books for Java developers")
.call()
.entity(
new ParameterizedTypeReference<List<Book>>() {
}
);
Because Java removes generic type information at runtime through type erasure, we use ParameterizedTypeReference to preserve the complete target type. Spring AI can then map the model response to a List<Book>.
Wrapper Records vs Collections
We can represent collections in two common ways.
The first approach is to wrap the collection inside another Java type.
public record Movies(
List<Movie> movies
) {
}
Then we can map the response directly to Movies.
Movies movies = chatClient.prompt()
.user("Recommend 5 movies")
.call()
.entity(Movies.class);
Alternatively, we can map the response directly to a List<Movie>.
List<Movie> movies = chatClient.prompt()
.user("Recommend 5 movies")
.call()
.entity(
new ParameterizedTypeReference<List<Movie>>() {
}
);
Both approaches are valid. A wrapper record can be useful when the response may contain additional information.
For example:
public record Movies(
List<Movie> movies,
String genre,
int count
) {
}
A direct List<Movie> is convenient when we only need the collection.
Externalized Prompts with Structured Output
In the previous article, we learned how to externalize prompts into String Template files.
We can use the same approach with structured output.
Let's create two prompt files.
src/main/resources/prompts
├── system.st
└── user.st
system.st
You are an experienced movie recommendation assistant.
Your responsibility is to recommend movies that closely match the user's
preferences.
Follow these rules:
1. Recommend real movies only.
2. Do not invent movie titles.
3. Match the requested genre and language as closely as possible.
4. Avoid recommending the same movie more than once.
5. Write a concise summary for each movie.
6. Do not reveal plot twists, endings, or major spoilers.
7. Return exactly the number of movies requested by the user.
8. If the user's preferences are broad, provide a diverse selection.
9. Do not include additional commentary outside the requested response.
user.st
Recommend {count} movies based on the following preferences:
Genre: {genre}
Preferred language: {language}
For every movie, provide:
- The movie title
- Its primary genre
- Year of release
- A short, spoiler-free summary
The prompt defines what the model should recommend.
Our Java type defines the structure expected by the application.
public record Movie(
String title,
String genre,
int year,
String language,
String summary
) {
}
Putting It All Together
Let's put the concepts we discussed into a complete example.
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.core.io.Resource;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.util.List;
@RestController
public class OutputController {
private final ChatClient chatClient;
@Value("classpath:/prompts/system.st")
private Resource systemPrompt;
@Value("classpath:/prompts/user.st")
private Resource userPrompt;
public OutputController(ChatClient chatClient) {
this.chatClient = chatClient;
}
@GetMapping("/extract")
public String extract() {
return chatClient.prompt()
.user("""
Extract customer information:
John Smith
john@example.com
+1 555-1234
""")
.call()
.content();
}
@GetMapping("/extract2")
public Customer extract2() {
return chatClient.prompt()
.user("""
Extract customer information:
John Smith
john@example.com
+1 555-1234
""")
.call()
.entity(Customer.class);
}
@GetMapping("/movie-chat")
public Movies movieChat(
@RequestParam String genre,
@RequestParam String language,
@RequestParam(defaultValue = "5") int count) {
return chatClient.prompt()
.system(spec -> spec.text(systemPrompt))
.user(spec -> spec.text(userPrompt)
.param("genre", genre)
.param("language", language)
.param("count", count))
.call()
.entity(Movies.class);
}
@GetMapping("/movie-chat2")
public List<Movie> movieChat2(
@RequestParam String genre,
@RequestParam String language,
@RequestParam(defaultValue = "5") int count) {
return chatClient.prompt()
.system(spec -> spec.text(systemPrompt))
.user(spec -> spec.text(userPrompt)
.param("genre", genre)
.param("language", language)
.param("count", count))
.call()
.entity(
new ParameterizedTypeReference<List<Movie>>() {
}
);
}
public record Customer(
String name,
String email,
String phone
) {
}
public record Movies(
List<Movie> movies
) {
}
public record Movie(
String title,
String genre,
int year,
String language,
String summary
) {
}
}
- The
/extractendpoint returns the model response as free-form text. - The
/extract2endpoint maps the model response directly into aCustomerrecord. - The
/movie-chatendpoint demonstrates how a collection can be wrapped inside another Java record. - The
/movie-chat2maps the response directly into aList<Movie>usingParameterizedTypeReference.
Summary
In this article, we explored how Spring AI can map model responses directly into Java types using structured output.
We learned how to convert model responses into Java records and collections and how ParameterizedTypeReference can be used for generic types such as List<Movie>.
In the next article, we will explore conversation memory and learn how Spring AI maintains context across multiple user interactions.
Comments
Post a Comment