Class GoogleAiGeminiBatchChatModel

java.lang.Object

dev.langchain4j.model.googleai.GoogleAiGeminiBatchChatModel

@Experimental
public final class GoogleAiGeminiBatchChatModel
extends Object

Provides an interface for interacting with the Gemini Batch API, an asynchronous service designed for processing large volumes of requests at a reduced cost (50% of standard). It is ideal for non-urgent, large-scale tasks like data pre-processing or evaluations, with a Service Level Objective (SLO) of 24-hour turnaround, though completion is often much quicker.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	GoogleAiGeminiBatchChatModel.Builder

Method Summary

Modifier and Type	Method	Description
void	cancelBatchJob(BatchRequestResponse.BatchName name)	Cancels a batch operation that is currently pending or running.
BatchRequestResponse.BatchResponse<ChatResponse>	createBatchFromFile(String displayName, GeminiFiles.GeminiFile file)	Creates a batch of chat requests from an uploaded file.
BatchRequestResponse.BatchResponse<ChatResponse>	createBatchInline(String displayName, @Nullable Long priority, List<ChatRequest> requests)	Creates and enqueues a batch of content generation requests for asynchronous processing.
void	deleteBatchJob(BatchRequestResponse.BatchName name)	Deletes a batch job from the system.
BatchRequestResponse.BatchList<ChatResponse>	listBatchJobs(@Nullable Integer pageSize, @Nullable String pageToken)	Lists batch jobs with optional pagination.
BatchRequestResponse.BatchResponse<ChatResponse>	retrieveBatchResults(BatchRequestResponse.BatchName name)	Retrieves the current state and results of a batch operation.
void	writeBatchToFile(JsonLinesWriter writer, Iterable<BatchRequestResponse.BatchFileRequest<ChatRequest>> requests)	Writes a batch of chat requests to a JSONL file for later upload and processing.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

createBatchInline

public BatchRequestResponse.BatchResponse<ChatResponse> createBatchInline(String displayName,
 @Nullable Long priority,
 List<ChatRequest> requests)

Creates and enqueues a batch of content generation requests for asynchronous processing.

This method submits multiple chat requests as a single batch operation to the Gemini API. All requests in the batch must use the same model. The batch will be processed asynchronously, and the initial response will typically be in a BatchRequestResponse.BatchIncomplete state.

Batch processing offers a 50% cost reduction compared to real-time requests and has a 24-hour turnaround SLO, making it ideal for large-scale, non-urgent tasks.

Note: The inline API allows for a total request size of 20MB or under. Larger requests should use the File API

Parameters:

displayName - a user-defined name for the batch, used for identification

priority - optional priority for the batch; batches with higher priority values are processed before those with lower values; negative values are allowed; defaults to 0 if null

requests - a list of chat requests to be processed in the batch; all requests must use the same model

Returns:

a BatchRequestResponse.BatchResponse representing the initial state of the batch operation, typically BatchRequestResponse.BatchIncomplete

Throws:

IllegalArgumentException - if the requests contain different models

createBatchFromFile

public BatchRequestResponse.BatchResponse<ChatResponse> createBatchFromFile(String displayName,
 GeminiFiles.GeminiFile file)

Creates a batch of chat requests from an uploaded file.

This method allows you to create a batch job using a JSONL file that has been previously uploaded to the Gemini Files API. This is useful for larger batches that exceed the 20 MB inline request limit.

The file must contain batch requests in JSONL format, where each line is a JSON object with a "key" and "request" field. You can use writeBatchToFile(JsonLinesWriter, Iterable) to create properly formatted JSONL files.

Parameters:

displayName - a user-defined name for the batch, used for identification

file - the GeminiFile object representing the uploaded file containing batch requests

Returns:

a BatchRequestResponse.BatchResponse representing the initial state of the batch operation, typically BatchRequestResponse.BatchIncomplete

See Also:

• writeBatchToFile(JsonLinesWriter, Iterable)
• GeminiFiles.uploadFile(java.nio.file.Path, String)

writeBatchToFile

public void writeBatchToFile(JsonLinesWriter writer,
 Iterable<BatchRequestResponse.BatchFileRequest<ChatRequest>> requests)
                      throws IOException

Writes a batch of chat requests to a JSONL file for later upload and processing.

This method serializes chat requests into JSONL (JSON Lines) format, where each line contains a single request wrapped in a BatchRequestResponse.BatchFileRequest with a unique key. The resulting file can be uploaded using the Gemini Files API and then used to create a batch job via createBatchFromFile(String, GeminiFile).

Each request is converted to the internal Gemini format before being written to the file.

Example usage:

Path batchFile = Files.createTempFile("batch", ".jsonl");
try (JsonLinesWriter writer = new StreamingJsonLinesWriter(batchFile)) {
    List<BatchFileRequest<ChatRequest>> requests = List.of(
        new BatchFileRequest<>("request-1", ChatRequest.builder()
            .messages(UserMessage.from("Question 1"))
            .build()),
        new BatchFileRequest<>("request-2", ChatRequest.builder()
            .messages(UserMessage.from("Question 2"))
            .build())
    );
    batchModel.writeBatchToFile(writer, requests);
}

Parameters:

writer - the JsonLinesWriter to which the batch requests will be written

requests - an iterable collection of BatchFileRequest objects containing ChatRequest instances, each with a unique key identifier

Throws:

IOException - if an I/O error occurs while writing to the writer

See Also:

• createBatchFromFile(String, GeminiFile)
• JsonLinesWriter

retrieveBatchResults

public BatchRequestResponse.BatchResponse<ChatResponse> retrieveBatchResults(BatchRequestResponse.BatchName name)

Retrieves the current state and results of a batch operation.

This method polls the Gemini API to get the latest state of a previously created batch. The response can be:

• BatchRequestResponse.BatchIncomplete - if the batch is still pending or running
• BatchRequestResponse.BatchSuccess - if the batch completed successfully, containing all responses
• BatchRequestResponse.BatchError - if the batch failed, containing error details

Clients should poll this method at intervals to check the operation status until completion.

Parameters:

name - the name of the batch operation to retrieve, obtained from the initial createBatchInline(String, Long, List) call

Returns:

a BatchRequestResponse.BatchResponse representing the current state of the batch operation

cancelBatchJob

public void cancelBatchJob(BatchRequestResponse.BatchName name)

Cancels a batch operation that is currently pending or running.

This method attempts to cancel a batch job. Cancellation is only possible for batches that are in BatchRequestResponse.BatchJobState.BATCH_STATE_PENDING or BatchRequestResponse.BatchJobState.BATCH_STATE_RUNNING state. Batches that have already completed, failed, or been cancelled cannot be cancelled.

Parameters:

name - the name of the batch operation to cancel

Throws:

HttpException - if the batch cannot be cancelled (e.g., already completed, already cancelled, or does not exist)

deleteBatchJob

public void deleteBatchJob(BatchRequestResponse.BatchName name)

Deletes a batch job from the system.

This removes the batch job but does not cancel it if still running. Use cancelBatchJob(BatchName) to cancel a running batch.

Parameters:

name - the name of the batch job to delete

Throws:

RuntimeException - if the batch job cannot be deleted or does not exist

listBatchJobs

public BatchRequestResponse.BatchList<ChatResponse> listBatchJobs(@Nullable Integer pageSize,
 @Nullable String pageToken)

Lists batch jobs with optional pagination.

Parameters:

pageSize - the maximum number of batch jobs to return; if null, uses server default

pageToken - token for retrieving a specific page from BatchRequestResponse.BatchList.pageToken(); if null, returns the first page

Returns:

a BatchRequestResponse.BatchList containing batch responses and a token for the next page

Throws:

RuntimeException - if the server does not support this operation