Interface ChatModel

All Known Implementing Classes:

AnthropicChatModel, AzureOpenAiChatModel, BedrockChatModel, DisabledChatModel, GoogleAiGeminiChatModel, GoogleGenAiChatModel, GPULlama3ChatModel, HuggingFaceChatModel, JlamaChatModel, LocalAiChatModel, MistralAiChatModel, OllamaChatModel, OpenAiChatModel, OpenAiOfficialChatModel, OpenAiOfficialResponsesChatModel, OpenAiResponsesChatModel, VertexAiAnthropicChatModel, VertexAiChatModel, VertexAiGeminiChatModel, WatsonxChatModel, WatsonxDeploymentChatModel, WatsonxGatewayChatModel, WorkersAiChatModel

public interface ChatModel

Represents a language model that has a chat API.

See Also:

• StreamingChatModel

Method Summary

Modifier and Type	Method	Description
default ChatResponse	chat(ChatMessage... messages)
default ChatResponse	chat(ChatRequest chatRequest)	This is the main API to interact with the chat model.
default ChatResponse	chat(ChatRequest chatRequest, ChatRequestOptions options)	Sends a chat request with additional invocation options.
default String	chat(String userMessage)
default ChatResponse	chat(List<ChatMessage> messages)
default CompletableFuture<ChatResponse>	chatAsync(ChatMessage... messages)	Non-blocking convenience counterpart of chat(ChatMessage...).
default CompletableFuture<ChatResponse>	chatAsync(ChatRequest chatRequest)	Non-blocking counterpart of chat(ChatRequest): sends a chat request and returns a CompletableFuture that completes with the ChatResponse once the model responds.
default CompletableFuture<ChatResponse>	chatAsync(ChatRequest chatRequest, ChatRequestOptions options)	Sends a non-blocking chat request with additional invocation options.
default CompletableFuture<String>	chatAsync(String userMessage)	Non-blocking convenience counterpart of chat(String): completes with the text of the model's response.
default CompletableFuture<ChatResponse>	chatAsync(List<ChatMessage> messages)	Non-blocking convenience counterpart of chat(List).
default ChatRequestParameters	defaultRequestParameters()
default ChatResponse	doChat(ChatRequest chatRequest)
default CompletableFuture<ChatResponse>	doChatAsync(ChatRequest chatRequest)	SPI hook for a genuinely non-blocking chat implementation, invoked by chatAsync(ChatRequest).
default List<ChatModelListener>	listeners()
default ModelProvider	provider()
default Set<Capability>	supportedCapabilities()

Method Details

chat

default ChatResponse chat(ChatRequest chatRequest)

This is the main API to interact with the chat model.

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

Returns:

a ChatResponse, containing all the outputs from the LLM

chat

default ChatResponse chat(ChatRequest chatRequest,
 ChatRequestOptions options)

Sends a chat request with additional invocation options.

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

options - a ChatRequestOptions carrying listener attributes and other per-call metadata

Returns:

a ChatResponse, containing all the outputs from the LLM

Since:

1.13.0

doChat

default ChatResponse doChat(ChatRequest chatRequest)

chatAsync

@Experimental
default CompletableFuture<ChatResponse> chatAsync(ChatRequest chatRequest)

Non-blocking counterpart of chat(ChatRequest): sends a chat request and returns a CompletableFuture that completes with the ChatResponse once the model responds.

Unlike chat(ChatRequest), this method does not block the calling thread. Operational failures (including unsupported-parameter validation) are delivered through the returned future (completed exceptionally), not thrown synchronously — the async analog of how the publisher API signals errors via onError.

Registered ChatModelListeners are invoked: onRequest when the request is initiated, then onResponse once the response is available, or onError on failure.

Threading. The returned future is completed on the model's own thread - for HTTP models, the transport's I/O worker that reads the response (the JDK HTTP client's HttpClient-* workers). A continuation attached without an explicit executor (thenApply, thenAccept, ...) therefore runs on that thread, where blocking is as harmful as blocking in a ChatModelListener callback: it stalls the worker and, under concurrency, degrades throughput for every in-flight call. Keep continuations non-blocking, or hand blocking work to your own Executor via the *Async variants (thenApplyAsync(fn, executor)).

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

Returns:

a CompletableFuture of the ChatResponse

Since:

1.20.0

chatAsync

@Experimental
default CompletableFuture<ChatResponse> chatAsync(ChatRequest chatRequest,
 ChatRequestOptions options)

Sends a non-blocking chat request with additional invocation options.

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

options - a ChatRequestOptions carrying listener attributes and other per-call metadata

Returns:

a CompletableFuture of the ChatResponse

Since:

1.20.0

See Also:

• chatAsync(ChatRequest)

doChatAsync

@Experimental
default CompletableFuture<ChatResponse> doChatAsync(ChatRequest chatRequest)

SPI hook for a genuinely non-blocking chat implementation, invoked by chatAsync(ChatRequest).

The default returns a failed future carrying AsyncNotSupportedException to signal that this model has no native asynchronous implementation. Callers on the asynchronous and reactive path (for example the non-blocking RAG stages) detect this and either offload the blocking doChat(ChatRequest) or fail loudly with an actionable message. A model backed by remote HTTP I/O overrides this with a genuinely asynchronous call (no thread parked).

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

Returns:

a CompletableFuture of the ChatResponse

Since:

1.20.0

defaultRequestParameters

default ChatRequestParameters defaultRequestParameters()

listeners

default List<ChatModelListener> listeners()

provider

default ModelProvider provider()

chat

default String chat(String userMessage)

chat

default ChatResponse chat(ChatMessage... messages)

chat

default ChatResponse chat(List<ChatMessage> messages)

chatAsync

@Experimental
default CompletableFuture<String> chatAsync(String userMessage)

Non-blocking convenience counterpart of chat(String): completes with the text of the model's response.

Since:

1.20.0

chatAsync

@Experimental
default CompletableFuture<ChatResponse> chatAsync(ChatMessage... messages)

Non-blocking convenience counterpart of chat(ChatMessage...).

Since:

1.20.0

chatAsync

@Experimental
default CompletableFuture<ChatResponse> chatAsync(List<ChatMessage> messages)

Non-blocking convenience counterpart of chat(List).

Since:

1.20.0

supportedCapabilities

default Set<Capability> supportedCapabilities()