Interface StreamingChatModel

All Known Implementing Classes:

AnthropicStreamingChatModel, AzureOpenAiStreamingChatModel, BedrockStreamingChatModel, DisabledStreamingChatModel, GitHubModelsStreamingChatModel, GoogleAiGeminiStreamingChatModel, GPULlama3StreamingChatModel, JlamaStreamingChatModel, LocalAiStreamingChatModel, MistralAiStreamingChatModel, OllamaStreamingChatModel, OpenAiOfficialResponsesStreamingChatModel, OpenAiOfficialStreamingChatModel, OpenAiResponsesStreamingChatModel, OpenAiStreamingChatModel, VertexAiAnthropicStreamingChatModel, VertexAiGeminiStreamingChatModel, WatsonxStreamingChatModel

public interface StreamingChatModel

Represents a language model that has a chat API and can stream a response one token at a time.

See Also:

• ChatModel

Method Summary

Modifier and Type	Method	Description
default void	chat(ChatRequest request, ChatRequestOptions options, StreamingChatResponseHandler handler)	Sends a streaming chat request with additional invocation options.
default void	chat(ChatRequest request, StreamingChatResponseHandler handler)	This is the main API to interact with the chat model.
default void	chat(String userMessage, StreamingChatResponseHandler handler)
default void	chat(List<ChatMessage> messages, StreamingChatResponseHandler handler)
default ChatRequestParameters	defaultRequestParameters()
default void	doChat(ChatRequest chatRequest, StreamingChatResponseHandler handler)
default List<ChatModelListener>	listeners()
default ModelProvider	provider()
default Set<Capability>	supportedCapabilities()

Method Details

chat

default void chat(ChatRequest request,
 StreamingChatResponseHandler handler)

This is the main API to interact with the chat model.

Parameters:

request - a ChatRequest, containing all the inputs to the LLM

handler - a StreamingChatResponseHandler that will handle streaming response from the LLM

chat

default void chat(ChatRequest request,
 ChatRequestOptions options,
 StreamingChatResponseHandler handler)

Sends a streaming chat request with additional invocation options.

Parameters:

request - a ChatRequest, containing all the inputs to the LLM

options - a ChatRequestOptions carrying listener attributes and other per-call metadata

handler - a StreamingChatResponseHandler that will handle streaming response from the LLM

Since:

1.13.0

doChat

default void doChat(ChatRequest chatRequest,
 StreamingChatResponseHandler handler)

defaultRequestParameters

default ChatRequestParameters defaultRequestParameters()

listeners

default List<ChatModelListener> listeners()

provider

default ModelProvider provider()

chat

default void chat(String userMessage,
 StreamingChatResponseHandler handler)

chat

default void chat(List<ChatMessage> messages,
 StreamingChatResponseHandler handler)

supportedCapabilities

default Set<Capability> supportedCapabilities()