Skip to main content

Anthropic

Maven Dependency​

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-anthropic</artifactId>
    <version>1.4.0</version>
</dependency>

AnthropicChatModel​

AnthropicChatModel model = AnthropicChatModel.builder()
    .apiKey(System.getenv("ANTHROPIC_API_KEY"))
    .modelName(CLAUDE_3_5_SONNET_20240620)
    .build();
String answer = model.chat("Say 'Hello World'");
System.out.println(answer);

Customizing AnthropicChatModel​

AnthropicChatModel model = AnthropicChatModel.builder()
    .httpClientBuilder(...)
    .baseUrl(...)
    .apiKey(...)
    .version(...)
    .beta(...)
    .modelName(...)
    .temperature(...)
    .topP(...)
    .topK(...)
    .maxTokens(...)
    .stopSequences(...)
    .toolSpecifications(...)
    .toolChoice(...)
    .cacheSystemMessages(...)
    .cacheTools(...)
    .thinkingType(...)
    .thinkingBudgetTokens(...)
    .returnThinking(...)
    .sendThinking(...)
    .timeout(...)
    .maxRetries(...)
    .logRequests(...)
    .logResponses(...)
    .listeners(...)
    .defaultRequestParameters(...)
    .build();

See the description of some of the parameters above here.

AnthropicStreamingChatModel​

AnthropicStreamingChatModel model = AnthropicStreamingChatModel.builder()
    .apiKey(System.getenv("ANTHROPIC_API_KEY"))
    .modelName(CLAUDE_3_5_SONNET_20240620)
    .build();

model.chat("Say 'Hello World'", new StreamingChatResponseHandler() {

    @Override
    public void onPartialResponse(String partialResponse) {
        // this method is called when a new partial response is available. It can consist of one or more tokens.
    }

    @Override
    public void onCompleteResponse(ChatResponse completeResponse) {
        // this method is called when the model has completed responding
    }

    @Override
    public void onError(Throwable error) {
        // this method is called when an error occurs
    }
});

Customizing AnthropicStreamingChatModel​

Identical to the AnthropicChatModel, see above.

Tools​

Anthropic supports tools in both streaming and non-streaming mode.

Anthropic documentation on tools can be found here.

Caching​

AnthropicChatModel and AnthropicStreamingChatModel support caching of system messages and tools. Caching is disabled by default. It can be enabled by setting the cacheSystemMessages and cacheTools parameters, respectively.

When enabled,cache_control blocks will be added to the last system message and tool, respectively.

To use caching, please set beta("prompt-caching-2024-07-31").

AnthropicChatModel and AnthropicStreamingChatModel return AnthropicTokenUsage in response which contains cacheCreationInputTokens and cacheReadInputTokens.

More info on caching can be found here.

Thinking​

Both AnthropicChatModel and AnthropicStreamingChatModel support thinking feature.

It is controlled by the following parameters:

  • thinkingType and thinkingBudgetTokens: enable thinking, see more details here.
  • returnThinking: controls whether to return thinking (if available) inside AiMessage.thinking() and whether to invoke StreamingChatResponseHandler.onPartialThinking() and TokenStream.onPartialThinking() callbacks when using BedrockStreamingChatModel. Disabled by default. If enabled, tinking signatures will also be stored and returned inside the AiMessage.attributes().
  • sendThinking: controls whether to send thinking and signatures stored in AiMessage to the LLM in follow-up requests. Enabled by default.

Here is an example of how to configure thinking:

ChatModel model = AnthropicChatModel.builder()
        .apiKey(System.getenv("ANTHROPIC_API_KEY"))
        .modelName(CLAUDE_3_7_SONNET_20250219)
        .thinkingType("enabled")
        .thinkingBudgetTokens(1024)
        .maxTokens(1024 + 100)
        .returnThinking(true)
        .sendThinking(true)
        .build();

AnthropicTokenCountEstimator​

TokenCountEstimator tokenCountEstimator = AnthropicTokenCountEstimator.builder()
        .modelName(CLAUDE_3_OPUS_20240229)
        .apiKey(System.getenv("ANTHROPIC_API_KEY"))
        .logRequests(true)
        .logResponses(true)
        .build();

List<ChatMessage> messages = List.of(...);

int tokenCount = tokenCountEstimator.estimateTokenCountInMessages(messages);

Quarkus​

See more details here.

Spring Boot​

Import Spring Boot starter for Anthropic:

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-anthropic-spring-boot-starter</artifactId>
    <version>1.4.0-beta10</version>
</dependency>

Configure AnthropicChatModel bean:

langchain4j.anthropic.chat-model.api-key = ${ANTHROPIC_API_KEY}

Configure AnthropicStreamingChatModel bean:

langchain4j.anthropic.streaming-chat-model.api-key = ${ANTHROPIC_API_KEY}

Examples​