Class AnthropicBatchChatModel

java.lang.Object
dev.langchain4j.model.anthropic.AnthropicBatchChatModel
All Implemented Interfaces:
BatchChatModel
@Experimental public final class AnthropicBatchChatModel extends Object implements BatchChatModel
A BatchChatModel for the Anthropic Message Batches API, which processes multiple chat requests asynchronously at 50% of the standard per-token price.

Each submitted ChatRequest is mapped to a request identical to the one AnthropicChatModel would send, so per-request parameters (model, tools, thinking, caching, etc.) behave the same in a batch as in a single call. Requests are submitted inline; the returned BatchResponse carries the batch id and its current BatchState. Results are fetched by retrieve(String) once the batch has ended, preserving submission order.

See Also:

  • Constructor Details

  • Method Details

    • submit

      public BatchResponse<ChatResponse> submit(BatchRequest<ChatRequest> request)
      Creates a batch of chat requests and submits them for asynchronous processing.

      The returned BatchResponse represents the status of the batch operation.

      All requests are submitted inline in a single HTTP request, so Anthropic's limits of 100,000 requests and 256 MB per batch apply. Exceeding either is rejected by the API.

      Specified by:
      submit in interface BatchChatModel
      Parameters:
      request - the list of chat requests to process in the batch
      Returns:
      a BatchResponse representing the initial state of the batch operation

    • retrieve

      public BatchResponse<ChatResponse> retrieve(String batchId)
      Retrieves the current state and results of a chat batch operation.

      The response indicates whether the batch is still processing, completed successfully, or failed. Clients should poll this method at intervals until the batch completes.

      Once the batch has ended, all of its results are downloaded and held in memory. For very large batches (Anthropic allows up to 100,000 requests per batch) this can require a correspondingly large heap.

      Specified by:
      retrieve in interface BatchChatModel
      Parameters:
      batchId - the batch identifier obtained from BatchChatModel.submit(BatchRequest)
      Returns:
      a BatchResponse representing the current state of the chat batch operation

    • cancel

      public void cancel(String batchId)
      Description copied from interface: BatchChatModel
      Cancels a chat batch operation that is currently pending or running.
      Specified by:
      cancel in interface BatchChatModel
      Parameters:
      batchId - the batch identifier to cancel

    • list

      public BatchPage<ChatResponse> list(@Nullable BatchPagination pagination)
      Description copied from interface: BatchChatModel
      Lists chat batch jobs with optional pagination.
      Specified by:
      list in interface BatchChatModel
      Parameters:
      pagination - the maximum number of batch jobs to return and token for retrieving a specific page; if null, uses server default
      Returns:
      a BatchPage containing chat batch responses and pagination information

    • builder

      public static AnthropicBatchChatModel.Builder builder()
      Returns:
      a new AnthropicBatchChatModel.Builder for AnthropicBatchChatModel.