Interface BatchChatModel

All Known Implementing Classes:
GoogleAiGeminiBatchChatModel
@Experimental public interface BatchChatModel
A model interface for processing multiple chat requests asynchronously in a batch.

Batch processing typically offers significant cost reductions compared to real-time chat requests and is ideal for large-scale, non-urgent conversational or instruction-following tasks.

See Also:

  • Method Details

    • submit

      BatchResponse<ChatResponse> submit(BatchRequest<ChatRequest> request)
      Creates a batch of chat requests and submits them for asynchronous processing.

      The returned BatchResponse represents the status of the batch operation.

      Parameters:
      request - the list of chat requests to process in the batch
      Returns:
      a BatchResponse representing the initial state of the batch operation

    • retrieve

      BatchResponse<ChatResponse> retrieve(String batchId)
      Retrieves the current state and results of a chat batch operation.

      The response indicates whether the batch is still processing, completed successfully, or failed. Clients should poll this method at intervals until the batch completes.

      Parameters:
      batchId - the batch identifier obtained from submit(BatchRequest)
      Returns:
      a BatchResponse representing the current state of the chat batch operation

    • cancel

      void cancel(String batchId)
      Cancels a chat batch operation that is currently pending or running.
      Parameters:
      batchId - the batch identifier to cancel

    • list

      BatchPage<ChatResponse> list(@Nullable BatchPagination pagination)
      Lists chat batch jobs with optional pagination.
      Parameters:
      pagination - the maximum number of batch jobs to return and token for retrieving a specific page; if null, uses server default
      Returns:
      a BatchPage containing chat batch responses and pagination information