Package dev.langchain4j.model.openai
Class OpenAiStreamingChatModel
java.lang.Object
dev.langchain4j.model.openai.OpenAiStreamingChatModel
- All Implemented Interfaces:
StreamingChatLanguageModel
,TokenCountEstimator
public class OpenAiStreamingChatModel
extends Object
implements StreamingChatLanguageModel, TokenCountEstimator
Represents an OpenAI language model with a chat completion interface, such as gpt-3.5-turbo and gpt-4.
The model's response is streamed token by token and should be handled with
StreamingResponseHandler
.
You can find description of parameters here.-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic class
-
Constructor Summary
ConstructorsConstructorDescriptionOpenAiStreamingChatModel
(String baseUrl, String apiKey, String organizationId, String modelName, Double temperature, Double topP, List<String> stop, Integer maxTokens, Double presencePenalty, Double frequencyPenalty, Map<String, Integer> logitBias, String responseFormat, Integer seed, String user, Boolean strictTools, Boolean parallelToolCalls, Duration timeout, Proxy proxy, Boolean logRequests, Boolean logResponses, Tokenizer tokenizer, Map<String, String> customHeaders, List<ChatModelListener> listeners) -
Method Summary
Modifier and TypeMethodDescriptionbuilder()
int
estimateTokenCount
(List<ChatMessage> messages) Estimates the count of tokens in the specified list of messages.void
generate
(List<ChatMessage> messages, ToolSpecification toolSpecification, StreamingResponseHandler<AiMessage> handler) Generates a response from the model based on a list of messages and a tool specification.void
generate
(List<ChatMessage> messages, StreamingResponseHandler<AiMessage> handler) Generates a response from the model based on a sequence of messages.void
generate
(List<ChatMessage> messages, List<ToolSpecification> toolSpecifications, StreamingResponseHandler<AiMessage> handler) Generates a response from the model based on a list of messages and a list of tool specifications.static OpenAiStreamingChatModel
withApiKey
(String apiKey) Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface dev.langchain4j.model.chat.StreamingChatLanguageModel
generate, generate
Methods inherited from interface dev.langchain4j.model.chat.TokenCountEstimator
estimateTokenCount, estimateTokenCount, estimateTokenCount, estimateTokenCount
-
Constructor Details
-
OpenAiStreamingChatModel
public OpenAiStreamingChatModel(String baseUrl, String apiKey, String organizationId, String modelName, Double temperature, Double topP, List<String> stop, Integer maxTokens, Double presencePenalty, Double frequencyPenalty, Map<String, Integer> logitBias, String responseFormat, Integer seed, String user, Boolean strictTools, Boolean parallelToolCalls, Duration timeout, Proxy proxy, Boolean logRequests, Boolean logResponses, Tokenizer tokenizer, Map<String, String> customHeaders, List<ChatModelListener> listeners)
-
-
Method Details
-
modelName
-
generate
Description copied from interface:StreamingChatLanguageModel
Generates a response from the model based on a sequence of messages. Typically, the sequence contains messages in the following order: System (optional) - User - AI - User - AI - User ...- Specified by:
generate
in interfaceStreamingChatLanguageModel
- Parameters:
messages
- A list of messages.handler
- The handler for streaming the response.
-
generate
public void generate(List<ChatMessage> messages, List<ToolSpecification> toolSpecifications, StreamingResponseHandler<AiMessage> handler) Description copied from interface:StreamingChatLanguageModel
Generates a response from the model based on a list of messages and a list of tool specifications. The response may either be a text message or a request to execute one of the specified tools. Typically, the list contains messages in the following order: System (optional) - User - AI - User - AI - User ...- Specified by:
generate
in interfaceStreamingChatLanguageModel
- Parameters:
messages
- A list of messages.toolSpecifications
- A list of tools that the model is allowed to execute. The model autonomously decides whether to use any of these tools.handler
- The handler for streaming the response.AiMessage
can contain either a textual response or a request to execute one of the tools.
-
generate
public void generate(List<ChatMessage> messages, ToolSpecification toolSpecification, StreamingResponseHandler<AiMessage> handler) Description copied from interface:StreamingChatLanguageModel
Generates a response from the model based on a list of messages and a tool specification.- Specified by:
generate
in interfaceStreamingChatLanguageModel
- Parameters:
messages
- A list of messages.toolSpecification
- A tool that the model is allowed to execute.handler
- The handler for streaming the response.
-
estimateTokenCount
Description copied from interface:TokenCountEstimator
Estimates the count of tokens in the specified list of messages.- Specified by:
estimateTokenCount
in interfaceTokenCountEstimator
- Parameters:
messages
- the list of messages- Returns:
- the estimated count of tokens
-
withApiKey
-
builder
-