Package dev.langchain4j.model.chat

Interface StreamingChatLanguageModel

All Known Implementing Classes:

AbstractBedrockStreamingChatModel, AnthropicStreamingChatModel, AzureOpenAiStreamingChatModel, BedrockAnthropicStreamingChatModel, DisabledStreamingChatLanguageModel, GitHubModelsStreamingChatModel, GoogleAiGeminiStreamingChatModel, JlamaStreamingChatModel, LocalAiStreamingChatModel, MistralAiStreamingChatModel, OllamaStreamingChatModel, OpenAiStreamingChatModel, VertexAiGeminiStreamingChatModel

public interface StreamingChatLanguageModel

Represents a language model that has a chat API and can stream a response one token at a time.

See Also:

• ChatLanguageModel

Method Summary

Modifier and Type	Method	Description
default void	chat(ChatRequest chatRequest, StreamingChatResponseHandler handler)	This is the main API to interact with the chat model.
default void	chat(String userMessage, StreamingChatResponseHandler handler)
default void	chat(List<ChatMessage> messages, StreamingChatResponseHandler handler)
default ChatRequestParameters	defaultRequestParameters()
default void	doChat(ChatRequest chatRequest, StreamingChatResponseHandler handler)
default void	generate(UserMessage userMessage, StreamingResponseHandler<AiMessage> handler)	Deprecated, for removal: This API element is subject to removal in a future version. please use chat(List, StreamingChatResponseHandler) instead
default void	generate(String userMessage, StreamingResponseHandler<AiMessage> handler)	Deprecated, for removal: This API element is subject to removal in a future version. please use chat(String, StreamingChatResponseHandler) instead
default void	generate(List<ChatMessage> messages, ToolSpecification toolSpecification, StreamingResponseHandler<AiMessage> handler)	Deprecated, for removal: This API element is subject to removal in a future version. please use chat(ChatRequest, StreamingChatResponseHandler) instead.
void	generate(List<ChatMessage> messages, StreamingResponseHandler<AiMessage> handler)	Deprecated, for removal: This API element is subject to removal in a future version. please use chat(List, StreamingChatResponseHandler) instead
default void	generate(List<ChatMessage> messages, List<ToolSpecification> toolSpecifications, StreamingResponseHandler<AiMessage> handler)	Deprecated, for removal: This API element is subject to removal in a future version. please use chat(ChatRequest, StreamingChatResponseHandler) instead.
default List<ChatModelListener>	listeners()
default Set<Capability>	supportedCapabilities()

Method Details

chat

default void chat(ChatRequest chatRequest,
 StreamingChatResponseHandler handler)

This is the main API to interact with the chat model.

A temporary default implementation of this method is necessary until all StreamingChatLanguageModel implementations adopt it. It should be removed once that occurs.

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

handler - a StreamingChatResponseHandler that will handle streaming response from the LLM

chat

default void chat(String userMessage,
 StreamingChatResponseHandler handler)

chat

default void chat(List<ChatMessage> messages,
 StreamingChatResponseHandler handler)

listeners

default List<ChatModelListener> listeners()

doChat

default void doChat(ChatRequest chatRequest,
 StreamingChatResponseHandler handler)

defaultRequestParameters

default ChatRequestParameters defaultRequestParameters()

supportedCapabilities

default Set<Capability> supportedCapabilities()

generate

@Deprecated(forRemoval=true)
default void generate(String userMessage,
 StreamingResponseHandler<AiMessage> handler)

Deprecated, for removal: This API element is subject to removal in a future version.

please use chat(String, StreamingChatResponseHandler) instead

Generates a response from the model based on a message from a user.

Parameters:

userMessage - The message from the user.

handler - The handler for streaming the response.

generate

@Deprecated(forRemoval=true)
default void generate(UserMessage userMessage,
 StreamingResponseHandler<AiMessage> handler)

Deprecated, for removal: This API element is subject to removal in a future version.

please use chat(List, StreamingChatResponseHandler) instead

Generates a response from the model based on a message from a user.

Parameters:

userMessage - The message from the user.

handler - The handler for streaming the response.

generate

@Deprecated(forRemoval=true)
void generate(List<ChatMessage> messages,
 StreamingResponseHandler<AiMessage> handler)

Deprecated, for removal: This API element is subject to removal in a future version.

please use chat(List, StreamingChatResponseHandler) instead

Generates a response from the model based on a sequence of messages. Typically, the sequence contains messages in the following order: System (optional) - User - AI - User - AI - User ...

Parameters:

messages - A list of messages.

handler - The handler for streaming the response.

generate

@Deprecated(forRemoval=true)
default void generate(List<ChatMessage> messages,
 List<ToolSpecification> toolSpecifications,
 StreamingResponseHandler<AiMessage> handler)

Deprecated, for removal: This API element is subject to removal in a future version.

please use chat(ChatRequest, StreamingChatResponseHandler) instead. See ChatRequestParameters.toolSpecifications().

Generates a response from the model based on a list of messages and a list of tool specifications. The response may either be a text message or a request to execute one of the specified tools. Typically, the list contains messages in the following order: System (optional) - User - AI - User - AI - User ...

Parameters:

messages - A list of messages.

toolSpecifications - A list of tools that the model is allowed to execute. The model autonomously decides whether to use any of these tools.

handler - The handler for streaming the response. AiMessage can contain either a textual response or a request to execute one of the tools.

Throws:

UnsupportedFeatureException - if tools are not supported by the underlying LLM API

generate

@Deprecated(forRemoval=true)
default void generate(List<ChatMessage> messages,
 ToolSpecification toolSpecification,
 StreamingResponseHandler<AiMessage> handler)

Deprecated, for removal: This API element is subject to removal in a future version.

please use chat(ChatRequest, StreamingChatResponseHandler) instead. See ChatRequestParameters.toolSpecifications() and ChatRequestParameters.toolChoice().

Generates a response from the model based on a list of messages and a single tool specification. The model is forced to execute the specified tool. This is usually achieved by setting `tool_choice=ANY` in the LLM provider API.

Parameters:

messages - A list of messages.

toolSpecification - The specification of a tool that must be executed. The model is forced to execute this tool.

handler - The handler for streaming the response.

Throws:

UnsupportedFeatureException - if tools are not supported by the underlying LLM API