Building a Universal Java Client for Large Language Models

In today's rapidly evolving AI landscape, developers often need to work with multiple Large Language Model (LLM) providers to find the best solution for their specific use case. Whether you're exploring OpenAI's GPT models, Anthropic's Claude, or running local models via Ollama, having a unified interface can significantly simplify development and make it easier to switch between providers.

The Java LLM Client project provides exactly this: a clean, consistent API for interacting with various LLM providers through a single library. Let's explore how this library works and how you can use it in your Java applications.

Core Features

The library offers several key features that make working with LLMs easier:

1. Unified Interface: Interact with different LLM providers through a consistent API
2. Multiple Provider Support: Currently supports OpenAI, Anthropic, Google, Groq, and Ollama
3. Chat Completions: Send messages and receive responses from language models
4. Embeddings: Generate vector representations of text where supported
5. Factory Pattern: Easily create service instances for different providers

Architecture Overview

The library is built around a few key interfaces and classes:

• GenerativeAIService: The main interface for interacting with LLMs
• GenerativeAIFactory: Factory interface for creating service instances
• GenerativeAIDriverManager: Registry that manages available services
• Provider-specific implementations in separate packages

This design follows the classic factory pattern, allowing you to:

1. Register service factories with the GenerativeAIDriverManager
2. Create service instances through the manager
3. Use a consistent API to interact with different providers

Getting Started

To use the library, first add it to your Maven project:

xml
<dependency>
    <groupId>org.llm</groupId>
    <artifactId>llmapi</artifactId>
    <version>1.0.0</version>
</dependency>

Basic Usage Example

Here's how to set up and use the library:

java
// Register service providers
GenerativeAIDriverManager.registerService(OpenAIFactory.NAME, new OpenAIFactory());
GenerativeAIDriverManager.registerService(AnthropicAIFactory.NAME, new AnthropicAIFactory());
// Register more providers as needed

// Create an OpenAI service
Map<String, Object> properties = Map.of("apiKey", System.getenv("gpt_key"));
var service = GenerativeAIDriverManager.create(
    OpenAIFactory.NAME, 
    "https://api.openai.com/", 
    properties
);

// Create and send a chat request
var message = new ChatMessage("user", "Hello, how are you?");
var conversation = new ChatRequest("gpt-4o-mini", List.of(message));
var reply = service.chat(conversation);
System.out.println(reply.message());

// Generate embeddings
var vector = service.embedding(
    new EmbeddingRequest("text-embedding-3-small", "How are you")
);
System.out.println(Arrays.toString(vector.embedding()));

Working with Different Providers

OpenAI

java
Map<String, Object> properties = Map.of("apiKey", System.getenv("gpt_key"));
var service = GenerativeAIDriverManager.create(
    OpenAIFactory.NAME, 
    "https://api.openai.com/", 
    properties
);

// Chat with GPT-4o mini
var conversation = new ChatRequest("gpt-4o-mini", 
    List.of(new ChatMessage("user", "Hello, how are you?")));
var reply = service.chat(conversation);

Anthropic

java
Map<String, Object> properties = Map.of("apiKey", System.getenv("ANTHROPIC_API_KEY"));
var service = GenerativeAIDriverManager.create(
    AnthropicAIFactory.NAME, 
    "https://api.anthropic.com", 
    properties
);

// Chat with Claude
var conversation = new ChatRequest("claude-3-7-sonnet-20250219", 
    List.of(new ChatMessage("user", "Hello, how are you?")));
var reply = service.chat(conversation);

Ollama (Local Models)

java
// No API key needed for local models
Map<String, Object> properties = Map.of();
var service = GenerativeAIDriverManager.create(
    OllamaFactory.NAME, 
    "http://localhost:11434", 
    properties
);

// Chat with locally hosted Llama model
var conversation = new ChatRequest("llama3.2", 
    List.of(new ChatMessage("user", "Hello, how are you?")));
var reply = service.chat(conversation);

Under the Hood

The library uses an RPC (Remote Procedure Call) client to handle the HTTP communication with various APIs. Each provider's implementation:

1. Creates appropriate request objects with the required format
2. Sends requests to the corresponding API endpoints
3. Parses responses into a consistent format
4. Handles errors gracefully

The RpcBuilder creates proxy instances of service interfaces, handling the HTTP communication details so you don't have to.

Supported Models

The library currently supports several models across different providers:

• OpenAI: all
• Anthropic: all
• Google: gemini-2.0-flash
• Groq: all
• Ollama: any other model you have locally

Extending the Library

One of the strengths of this design is how easily it can be extended to support new providers or features:

1. Create a new implementation of GenerativeAIFactory
2. Implement GenerativeAIService for the new provider
3. Create necessary request/response models
4. Register the new factory with GenerativeAIDriverManager

Conclusion

The Java LLM Client provides a clean, consistent way to work with multiple LLM providers in Java applications. By abstracting away the differences between APIs, it allows developers to focus on their application logic rather than the details of each provider's implementation.

Whether you're building a chatbot, generating embeddings for semantic search, or experimenting with different LLM providers, this library offers a straightforward way to integrate these capabilities into your Java applications.

The project's use of standard Java patterns like factories and interfaces makes it easy to understand and extend, while its modular design allows you to use only the providers you need. As the LLM ecosystem continues to evolve, this type of abstraction layer will become increasingly valuable for developers looking to build flexible, future-proof applications.

Link to github project - llmapi