Class GoogleAiGeminiBatchChatModel

java.lang.Object
dev.langchain4j.model.googleai.GoogleAiGeminiBatchChatModel
All Implemented Interfaces:
BatchChatModel
@Experimental public final class GoogleAiGeminiBatchChatModel extends Object implements BatchChatModel
Provides an interface for interacting with the Gemini Batch API, an asynchronous service designed for processing large volumes of requests at a reduced cost (50% of standard). It is ideal for non-urgent, large-scale tasks like data pre-processing or evaluations, with a Service Level Objective (SLO) of 24-hour turnaround, though completion is often much quicker.

Implements BatchChatModel for unified batch processing of chat requests.

See Also:

  • Method Details

    • submit

      public BatchResponse<ChatResponse> submit(BatchRequest<ChatRequest> request)
      Creates a batch of chat requests and submits them for asynchronous processing.

      The returned BatchResponse represents the status of the batch operation.

      Creates and enqueues a batch of content generation requests for asynchronous processing using default display name and priority. To set a custom display name or priority, pass a GeminiBatchRequest (it will resolve to submit(GeminiBatchRequest)).

      All requests are processed by the model configured on this batch model (via BaseGeminiChatModel.GoogleAiGeminiChatModelBaseBuilder.modelName(String)).

      Specified by:
      submit in interface BatchChatModel
      Parameters:
      request - a list of chat requests to be processed in the batch
      Returns:
      a BatchResponse representing the initial state of the batch operation

    • submit

      public BatchResponse<ChatResponse> submit(GeminiBatchRequest<ChatRequest> request)
      Creates and enqueues a batch of content generation requests for asynchronous processing, with Gemini-specific options such as display name and priority.
      Parameters:
      request - a GeminiBatchRequest carrying the chat requests and optional metadata
      Returns:
      a BatchResponse representing the initial state of the batch operation

    • submit

      public BatchResponse<ChatResponse> submit(String displayName, GeminiFiles.GeminiFile file)
      Creates a batch of chat requests from an uploaded file.

      This method allows you to create a batch job using a JSONL file that has been previously uploaded to the Gemini Files API. This is useful for larger batches that exceed the 20 MB inline request limit.

      The file must contain batch requests in JSONL format, where each line is a JSON object with a "key" and "request" field. You can use writeBatchToFile(JsonLinesWriter, Iterable) to create properly formatted JSONL files.

      Parameters:
      displayName - a user-defined name for the batch, used for identification
      file - the GeminiFile object representing the uploaded file containing batch requests
      Returns:
      a BatchResponse representing the initial state of the batch operation
      See Also:

    • writeBatchToFile

      public void writeBatchToFile(JsonLinesWriter writer, Iterable<BatchRequestResponse.BatchFileRequest<ChatRequest>> requests) throws IOException
      Writes a batch of chat requests to a JSONL file for later upload and processing.

      This method serializes chat requests into JSONL (JSON Lines) format, where each line contains a single request wrapped in a BatchRequestResponse.BatchFileRequest with a unique key. The resulting file can be uploaded using the Gemini Files API and then used to create a batch job via submit(String, GeminiFile).

      Each request is converted to the internal Gemini format before being written to the file.

      Example usage:

      Path batchFile = Files.createTempFile("batch", ".jsonl");
try (JsonLinesWriter writer = new StreamingJsonLinesWriter(batchFile)) {
    List<BatchFileRequest<ChatRequest>> requests = List.of(
        new BatchFileRequest<>("request-1", ChatRequest.builder()
            .messages(UserMessage.from("Question 1"))
            .build()),
        new BatchFileRequest<>("request-2", ChatRequest.builder()
            .messages(UserMessage.from("Question 2"))
            .build())
    );
    batchModel.writeBatchToFile(writer, requests);
}
      Parameters:
      writer - the JsonLinesWriter to which the batch requests will be written
      requests - an iterable collection of BatchFileRequest objects containing ChatRequest instances, each with a unique key identifier
      Throws:
      IOException - if an I/O error occurs while writing to the writer
      See Also:

    • retrieve

      public BatchResponse<ChatResponse> retrieve(String batchId)
      Retrieves the current state and results of a chat batch operation.

      The response indicates whether the batch is still processing, completed successfully, or failed. Clients should poll this method at intervals until the batch completes.

      Polls the Gemini API to get the latest state of a previously created batch. Clients should poll this method at intervals to check the operation status until completion.

      Specified by:
      retrieve in interface BatchChatModel
      Parameters:
      batchId - the batch name obtained from BatchChatModel.submit(BatchRequest) or submit(String, GeminiFile)
      Returns:
      a BatchResponse representing the current state of the batch operation

    • cancel

      public void cancel(String batchId)
      Cancels a chat batch operation that is currently pending or running.

      Cancellation is only possible for batches that are in BatchState.PENDING or BatchState.RUNNING state. Batches that have already completed, failed, or been cancelled cannot be cancelled.

      Specified by:
      cancel in interface BatchChatModel
      Parameters:
      batchId - the batch name to cancel
      Throws:
      HttpException - if the batch cannot be cancelled (e.g., already completed, already cancelled, or does not exist)

    • deleteBatchJob

      public void deleteBatchJob(String batchId)
      Deletes a batch job from the system.

      This removes the batch job record but does not cancel it if still running. Use cancel(String) to cancel a running batch before deletion.

      Parameters:
      batchId - the batch name to delete
      Throws:
      RuntimeException - if the batch job cannot be deleted or does not exist

    • list

      public BatchPage<ChatResponse> list(@Nullable BatchPagination batchPagination)
      Lists chat batch jobs with optional pagination.
      Specified by:
      list in interface BatchChatModel
      Parameters:
      batchPagination - the maximum number of batch jobs to return and token for retrieving a specific page; if null, uses server default
      Returns:
      a BatchPage containing chat batch responses and pagination information

    • builder

      public static GoogleAiGeminiBatchChatModel.Builder builder()
      Returns a new builder for constructing GoogleAiGeminiBatchChatModel instances.
      Returns:
      a new builder instance