Interface StreamingChatModel

All Known Implementing Classes:

AnthropicStreamingChatModel, AzureOpenAiStreamingChatModel, BedrockStreamingChatModel, DisabledStreamingChatModel, GitHubModelsStreamingChatModel, GoogleAiGeminiStreamingChatModel, GPULlama3StreamingChatModel, JlamaStreamingChatModel, LocalAiStreamingChatModel, MistralAiStreamingChatModel, OllamaStreamingChatModel, OpenAiOfficialStreamingChatModel, OpenAiStreamingChatModel, VertexAiGeminiStreamingChatModel, WatsonxStreamingChatModel

public interface StreamingChatModel

Represents a language model that has a chat API and can stream a response one token at a time.

See Also:

• ChatModel

Method Summary

Modifier and Type	Method	Description
default void	chat(ChatRequest chatRequest, StreamingChatResponseHandler handler)	This is the main API to interact with the chat model.
default void	chat(String userMessage, StreamingChatResponseHandler handler)
default void	chat(List<ChatMessage> messages, StreamingChatResponseHandler handler)
default ChatRequestParameters	defaultRequestParameters()
default void	doChat(ChatRequest chatRequest, StreamingChatResponseHandler handler)
default List<ChatModelListener>	listeners()
default ModelProvider	provider()
default Set<Capability>	supportedCapabilities()

Method Details

chat

default void chat(ChatRequest chatRequest,
 StreamingChatResponseHandler handler)

This is the main API to interact with the chat model.

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

handler - a StreamingChatResponseHandler that will handle streaming response from the LLM

doChat

default void doChat(ChatRequest chatRequest,
 StreamingChatResponseHandler handler)

defaultRequestParameters

default ChatRequestParameters defaultRequestParameters()

listeners

default List<ChatModelListener> listeners()

provider

default ModelProvider provider()

chat

default void chat(String userMessage,
 StreamingChatResponseHandler handler)

chat

default void chat(List<ChatMessage> messages,
 StreamingChatResponseHandler handler)

supportedCapabilities

default Set<Capability> supportedCapabilities()