In this post, we'll explore a Java interface designed to interact with Large Language Models (LLMs) in a type-safe manner. We'll break down the GenerativeAIService interface and its supporting classes to understand how it provides a structured approach to AI interactions.
The Problem: Unstructured LLM Responses
When working with LLMs, responses typically come as unstructured text. This presents challenges when you need to extract specific data or integrate AI capabilities into enterprise applications that expect structured data.
For example, if you want an LLM to generate JSON data for your application, you'd need to:
- Parse the response text
- Extract the JSON portion
- Deserialize it into your application objects
- Handle parsing errors appropriately
This process can be error-prone and verbose when implemented across multiple parts of your application.
Enter GenerativeAIService
The GenerativeAIService interface provides a clean solution to this problem by offering methods that not only communicate with LLM APIs but also handle the parsing of responses into Java objects.
Let's look at the core interface:
javapublic interface GenerativeAIService {
ChatMessageReply chat(ChatRequest conversation);
default ChatRequest prepareRequest(ChatRequest conversation, Map<String, Object> params) {
return ParamPreparedRequest.prepare(conversation, params);
}
default <T> T chat(ChatRequest conversation, Class<T> returnType) {
return chat(conversation, returnType, (jsonContent, e) -> {
throw new RuntimeException("Failed to parse JSON: " + jsonContent, e);
}).get();
}
default <T> Optional<T> chat(ChatRequest conversation, Class<T> returnType, BiConsumer<String, Exception> onFailedParsing) {
var reply = chat(conversation);
return ChatMessageJsonParser.parse(reply, returnType, onFailedParsing);
}//Other methods
}
The interface provides three key capabilities:
- Basic Chat Functionality: The
chat(ChatRequest)method handles direct communication with the LLM and returns raw responses. - Type-Safe Responses: Overloaded
chat()methods accept aClass<T>parameter to specify the expected return type, allowing the service to automatically parse the LLM response into the desired Java class. - Robust Error Handling: Options to provide custom error handling logic when parsing fails.
How It Works
Behind the scenes, the ChatMessageJsonParser class does the heavy lifting:
javapublic static <T> Optional<T> parse(ChatMessageReply reply, Class<T> returnType, BiConsumer<String, Exception> onFailedParsing) {
var message = reply.message().trim();
var jsonContent = _extractMessage(message);
return _cast(returnType, onFailedParsing, jsonContent);
}
It:
- Extracts JSON content from the LLM's response (which may be wrapped in markdown code blocks)
- Uses Gson to deserialize the JSON into the requested type
- Handles parsing errors according to the provided error handler
Parameterised Prompts
The interface also supports parameterised prompts through the ParamPreparedRequest class:
javadefault ChatRequest prepareRequest(ChatRequest conversation, Map<String, Object> params) {
return ParamPreparedRequest.prepare(conversation, params);
}
This allows you to:
- Create template prompts with placeholders like
{{parameter_name}} - Fill those placeholders at runtime with a map of parameter values
- Validate that all required parameters are provided
Code Example: Using the Typed API
Here's how you might use this API in practice:
java// Define a data class for the structured responsepublic static class ProductSuggestion {
public String productName;
public String description;
public double price;
public List<String> features;
}// Create a parameterised prompt
String prompt = """
Suggest a {{product_type}} product with {{feature_count}} features. Reply in JSON format
<example>
{
"productName": "product name",
"description": "product description",
"price": 100.0,
"features": ["feature 1", "feature 2", "feature 3", "feature 4", "feature 5"]
}
</example>
""";
var request = new ChatRequest(
"gemini-2.0-flash",
0.7f,
List.of(new ChatRequest.ChatMessage("user",
prompt))
);
// Prepare with parameters
Map<String, Object> params = Map.of(
"product_type", "smart home",
"feature_count", 5
);
request = service.prepareRequest(request, params);
// Get typed response
var suggestion = service.chat(request, ProductSuggestion.class);
System.out.println(suggestion);
Benefits of a Typed LLM API
- Type Safety: Catch type mismatches at compile time rather than runtime.
- Clean Integration: Seamlessly incorporate AI capabilities into existing Java applications.
- Reduced Boilerplate: Consolidate JSON parsing and error handling logic in one place.
- Parameter Validation: Ensure all required prompt parameters are provided before making API calls.
- Flexible Error Handling: Customize how parsing errors are handled based on your application's needs.
Implementation Considerations
When implementing this interface for different AI providers, consider:
- JSON Mode/Structure Mode: Now a days LLM support JSON or Structure mode and that can used as compared to Prompt instruction.
- Response formats: Ensure your parser can handle the specific output formats of each provider.
Conclusion
By creating a strongly-typed interface for LLM interactions, we bridge the gap between the unstructured world of AI and the structured requirements of enterprise applications. This approach enables developers to leverage the power of large language models while maintaining the type safety and predictability.
The GenerativeAIService interface provides a foundation that can be extended to work with various AI providers while providing a consistent interface for application code. It represents a step toward making AI capabilities more accessible and manageable in traditional software development workflows.
Code for this post is available @ TypeSafety
