Package dev.langchain4j.model.openai

Class OpenAiStreamingLanguageModel

java.lang.Object

dev.langchain4j.model.openai.OpenAiStreamingLanguageModel

All Implemented Interfaces:

StreamingLanguageModel, TokenCountEstimator

public class OpenAiStreamingLanguageModel
extends Object
implements StreamingLanguageModel, TokenCountEstimator

Represents an OpenAI language model with a completion interface, such as gpt-3.5-turbo-instruct. The model's response is streamed token by token and should be handled with StreamingResponseHandler. However, it's recommended to use OpenAiStreamingChatModel instead, as it offers more advanced features like function calling, multi-turn conversations, etc.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	OpenAiStreamingLanguageModel.OpenAiStreamingLanguageModelBuilder

Constructor Summary

Constructors

Constructor	Description
OpenAiStreamingLanguageModel(String baseUrl, String apiKey, String organizationId, String modelName, Double temperature, Duration timeout, Proxy proxy, Boolean logRequests, Boolean logResponses, Tokenizer tokenizer, Map<String,String> customHeaders)

Method Summary

Modifier and Type	Method	Description
static OpenAiStreamingLanguageModel.OpenAiStreamingLanguageModelBuilder	builder()
int	estimateTokenCount(String prompt)	Estimates the count of tokens in the given text.
void	generate(String prompt, StreamingResponseHandler<String> handler)	Generates a response from the model based on a prompt.
String	modelName()
static OpenAiStreamingLanguageModel	withApiKey(String apiKey)	Deprecated, for removal: This API element is subject to removal in a future version. Please use builder() instead, and explicitly set the model name and, if necessary, other parameters.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.langchain4j.model.language.StreamingLanguageModel

generate

Methods inherited from interface dev.langchain4j.model.language.TokenCountEstimator

estimateTokenCount, estimateTokenCount

Constructor Details

OpenAiStreamingLanguageModel

public OpenAiStreamingLanguageModel(String baseUrl,
 String apiKey,
 String organizationId,
 String modelName,
 Double temperature,
 Duration timeout,
 Proxy proxy,
 Boolean logRequests,
 Boolean logResponses,
 Tokenizer tokenizer,
 Map<String,String> customHeaders)

Method Details

modelName

public String modelName()

generate

public void generate(String prompt,
 StreamingResponseHandler<String> handler)

Description copied from interface: StreamingLanguageModel

Generates a response from the model based on a prompt.

Specified by:

generate in interface StreamingLanguageModel

Parameters:

prompt - The prompt.

handler - The handler for streaming the response.

estimateTokenCount

public int estimateTokenCount(String prompt)

Description copied from interface: TokenCountEstimator

Estimates the count of tokens in the given text.

Specified by:

estimateTokenCount in interface TokenCountEstimator

Parameters:

prompt - the text.

Returns:

the estimated count of tokens.

withApiKey

@Deprecated(forRemoval=true)
public static OpenAiStreamingLanguageModel withApiKey(String apiKey)

Deprecated, for removal: This API element is subject to removal in a future version.

Please use builder() instead, and explicitly set the model name and, if necessary, other parameters. The default values for the model name and temperature will be removed in future releases!

builder

public static OpenAiStreamingLanguageModel.OpenAiStreamingLanguageModelBuilder builder()