Package dev.langchain4j.model.vertexai

Class VertexAiGeminiStreamingChatModel

java.lang.Object

dev.langchain4j.model.vertexai.VertexAiGeminiStreamingChatModel

All Implemented Interfaces:

StreamingChatLanguageModel, Closeable, AutoCloseable

public class VertexAiGeminiStreamingChatModel
extends Object
implements StreamingChatLanguageModel, Closeable

Represents a Google Vertex AI Gemini language model with a stream chat completion interface, such as gemini-pro. See details here.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	VertexAiGeminiStreamingChatModel.VertexAiGeminiStreamingChatModelBuilder

Constructor Summary

Constructors

Constructor	Description
VertexAiGeminiStreamingChatModel(com.google.cloud.vertexai.generativeai.GenerativeModel generativeModel, com.google.cloud.vertexai.api.GenerationConfig generationConfig)
VertexAiGeminiStreamingChatModel(String project, String location, String modelName, Float temperature, Integer maxOutputTokens, Integer topK, Float topP, String responseMimeType, com.google.cloud.vertexai.api.Schema responseSchema, Map<HarmCategory,SafetyThreshold> safetySettings, Boolean useGoogleSearch, String vertexSearchDatastore, ToolCallingMode toolCallingMode, List<String> allowedFunctionNames, Boolean logRequests, Boolean logResponses, List<ChatModelListener> listeners)

Method Summary

Modifier and Type	Method	Description
static VertexAiGeminiStreamingChatModel.VertexAiGeminiStreamingChatModelBuilder	builder()
void	close()
void	generate(List<ChatMessage> messages, ToolSpecification toolSpecification, StreamingResponseHandler<AiMessage> handler)	Generates a response from the model based on a list of messages and a single tool specification.
void	generate(List<ChatMessage> messages, StreamingResponseHandler<AiMessage> handler)	Generates a response from the model based on a sequence of messages.
void	generate(List<ChatMessage> messages, List<ToolSpecification> toolSpecifications, StreamingResponseHandler<AiMessage> handler)	Generates a response from the model based on a list of messages and a list of tool specifications.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface dev.langchain4j.model.chat.StreamingChatLanguageModel

generate, generate

Constructor Details

VertexAiGeminiStreamingChatModel

public VertexAiGeminiStreamingChatModel(String project,
 String location,
 String modelName,
 Float temperature,
 Integer maxOutputTokens,
 Integer topK,
 Float topP,
 String responseMimeType,
 com.google.cloud.vertexai.api.Schema responseSchema,
 Map<HarmCategory,SafetyThreshold> safetySettings,
 Boolean useGoogleSearch,
 String vertexSearchDatastore,
 ToolCallingMode toolCallingMode,
 List<String> allowedFunctionNames,
 Boolean logRequests,
 Boolean logResponses,
 List<ChatModelListener> listeners)

VertexAiGeminiStreamingChatModel

public VertexAiGeminiStreamingChatModel(com.google.cloud.vertexai.generativeai.GenerativeModel generativeModel,
 com.google.cloud.vertexai.api.GenerationConfig generationConfig)

Method Details

generate

public void generate(List<ChatMessage> messages,
 StreamingResponseHandler<AiMessage> handler)

Description copied from interface: StreamingChatLanguageModel

Generates a response from the model based on a sequence of messages. Typically, the sequence contains messages in the following order: System (optional) - User - AI - User - AI - User ...

Specified by:

generate in interface StreamingChatLanguageModel

Parameters:

messages - A list of messages.

handler - The handler for streaming the response.

generate

public void generate(List<ChatMessage> messages,
 List<ToolSpecification> toolSpecifications,
 StreamingResponseHandler<AiMessage> handler)

Description copied from interface: StreamingChatLanguageModel

Generates a response from the model based on a list of messages and a list of tool specifications. The response may either be a text message or a request to execute one of the specified tools. Typically, the list contains messages in the following order: System (optional) - User - AI - User - AI - User ...

Specified by:

generate in interface StreamingChatLanguageModel

Parameters:

messages - A list of messages.

toolSpecifications - A list of tools that the model is allowed to execute. The model autonomously decides whether to use any of these tools.

handler - The handler for streaming the response. AiMessage can contain either a textual response or a request to execute one of the tools.

generate

public void generate(List<ChatMessage> messages,
 ToolSpecification toolSpecification,
 StreamingResponseHandler<AiMessage> handler)

Description copied from interface: StreamingChatLanguageModel

Generates a response from the model based on a list of messages and a single tool specification. The model is forced to execute the specified tool. This is usually achieved by setting `tool_choice=ANY` in the LLM provider API.

Specified by:

generate in interface StreamingChatLanguageModel

Parameters:

messages - A list of messages.

toolSpecification - The specification of a tool that must be executed. The model is forced to execute this tool.

handler - The handler for streaming the response.

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

builder

public static VertexAiGeminiStreamingChatModel.VertexAiGeminiStreamingChatModelBuilder builder()