Package dev.langchain4j.model.openai

Class OpenAiStreamingChatModel

java.lang.Object

dev.langchain4j.model.openai.OpenAiStreamingChatModel

All Implemented Interfaces:

StreamingChatLanguageModel, TokenCountEstimator

public class OpenAiStreamingChatModel
extends Object
implements StreamingChatLanguageModel, TokenCountEstimator

Represents an OpenAI language model with a chat completion interface, such as gpt-3.5-turbo and gpt-4. The model's response is streamed token by token and should be handled with StreamingResponseHandler. You can find description of parameters here.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	OpenAiStreamingChatModel.OpenAiStreamingChatModelBuilder

Constructor Summary

Constructors

Constructor

Description

OpenAiStreamingChatModel(String baseUrl, String apiKey, String organizationId, ChatRequestParameters defaultRequestParameters, String modelName, Double temperature, Double topP, List<String> stop, Integer maxTokens, Integer maxCompletionTokens, Double presencePenalty, Double frequencyPenalty, Map<String,Integer> logitBias, String responseFormat, Boolean strictJsonSchema, Integer seed, String user, Boolean strictTools, Boolean parallelToolCalls, Boolean store, Map<String,String> metadata, String serviceTier, Duration timeout, Proxy proxy, Boolean logRequests, Boolean logResponses, Tokenizer tokenizer, Map<String,String> customHeaders, List<ChatModelListener> listeners)

Method Summary

Modifier and Type	Method	Description
static OpenAiStreamingChatModel.OpenAiStreamingChatModelBuilder	builder()
void	chat(ChatRequest chatRequest, StreamingChatResponseHandler handler)	This is the main API to interact with the chat model.
OpenAiChatRequestParameters	defaultRequestParameters()
int	estimateTokenCount(List<ChatMessage> messages)	Estimates the count of tokens in the specified list of messages.
void	generate(List<ChatMessage> messages, ToolSpecification toolSpecification, StreamingResponseHandler<AiMessage> handler)	Generates a response from the model based on a list of messages and a single tool specification.
void	generate(List<ChatMessage> messages, StreamingResponseHandler<AiMessage> handler)	Generates a response from the model based on a sequence of messages.
void	generate(List<ChatMessage> messages, List<ToolSpecification> toolSpecifications, StreamingResponseHandler<AiMessage> handler)	Generates a response from the model based on a list of messages and a list of tool specifications.
String	modelName()
static OpenAiStreamingChatModel	withApiKey(String apiKey)	Deprecated, for removal: This API element is subject to removal in a future version. Please use builder() instead, and explicitly set the model name and, if necessary, other parameters.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.langchain4j.model.chat.StreamingChatLanguageModel

generate, generate, supportedCapabilities

Methods inherited from interface dev.langchain4j.model.chat.TokenCountEstimator

estimateTokenCount, estimateTokenCount, estimateTokenCount, estimateTokenCount

Constructor Details

OpenAiStreamingChatModel

public OpenAiStreamingChatModel(String baseUrl,
 String apiKey,
 String organizationId,
 ChatRequestParameters defaultRequestParameters,
 String modelName,
 Double temperature,
 Double topP,
 List<String> stop,
 Integer maxTokens,
 Integer maxCompletionTokens,
 Double presencePenalty,
 Double frequencyPenalty,
 Map<String,Integer> logitBias,
 String responseFormat,
 Boolean strictJsonSchema,
 Integer seed,
 String user,
 Boolean strictTools,
 Boolean parallelToolCalls,
 Boolean store,
 Map<String,String> metadata,
 String serviceTier,
 Duration timeout,
 Proxy proxy,
 Boolean logRequests,
 Boolean logResponses,
 Tokenizer tokenizer,
 Map<String,String> customHeaders,
 List<ChatModelListener> listeners)

Method Details

modelName

public String modelName()

defaultRequestParameters

public OpenAiChatRequestParameters defaultRequestParameters()

Specified by:

defaultRequestParameters in interface StreamingChatLanguageModel

chat

public void chat(ChatRequest chatRequest,
 StreamingChatResponseHandler handler)

Description copied from interface: StreamingChatLanguageModel

This is the main API to interact with the chat model. All the existing generate(...) methods (see below) will be deprecated and removed before 1.0.0 release.

A temporary default implementation of this method is necessary until all StreamingChatLanguageModel implementations adopt it. It should be removed once that occurs.

Specified by:

chat in interface StreamingChatLanguageModel

Parameters:

chatRequest - a ChatRequest, containing all the inputs to the LLM

handler - a StreamingChatResponseHandler that will handle streaming response from the LLM

generate

public void generate(List<ChatMessage> messages,
 StreamingResponseHandler<AiMessage> handler)

Description copied from interface: StreamingChatLanguageModel

Generates a response from the model based on a sequence of messages. Typically, the sequence contains messages in the following order: System (optional) - User - AI - User - AI - User ...

Specified by:

generate in interface StreamingChatLanguageModel

Parameters:

messages - A list of messages.

handler - The handler for streaming the response.

generate

public void generate(List<ChatMessage> messages,
 List<ToolSpecification> toolSpecifications,
 StreamingResponseHandler<AiMessage> handler)

Description copied from interface: StreamingChatLanguageModel

Generates a response from the model based on a list of messages and a list of tool specifications. The response may either be a text message or a request to execute one of the specified tools. Typically, the list contains messages in the following order: System (optional) - User - AI - User - AI - User ...

Specified by:

generate in interface StreamingChatLanguageModel

Parameters:

messages - A list of messages.

toolSpecifications - A list of tools that the model is allowed to execute. The model autonomously decides whether to use any of these tools.

handler - The handler for streaming the response. AiMessage can contain either a textual response or a request to execute one of the tools.

generate

public void generate(List<ChatMessage> messages,
 ToolSpecification toolSpecification,
 StreamingResponseHandler<AiMessage> handler)

Description copied from interface: StreamingChatLanguageModel

Generates a response from the model based on a list of messages and a single tool specification. The model is forced to execute the specified tool. This is usually achieved by setting `tool_choice=ANY` in the LLM provider API.

Specified by:

generate in interface StreamingChatLanguageModel

Parameters:

messages - A list of messages.

toolSpecification - The specification of a tool that must be executed. The model is forced to execute this tool.

handler - The handler for streaming the response.

estimateTokenCount

public int estimateTokenCount(List<ChatMessage> messages)

Description copied from interface: TokenCountEstimator

Estimates the count of tokens in the specified list of messages.

Specified by:

estimateTokenCount in interface TokenCountEstimator

Parameters:

messages - the list of messages

Returns:

the estimated count of tokens

withApiKey

@Deprecated(forRemoval=true)
public static OpenAiStreamingChatModel withApiKey(String apiKey)

Deprecated, for removal: This API element is subject to removal in a future version.

Please use builder() instead, and explicitly set the model name and, if necessary, other parameters. The default values for the model name and temperature will be removed in future releases!

builder

public static OpenAiStreamingChatModel.OpenAiStreamingChatModelBuilder builder()