Skip to main content

Anthropic

Maven Dependency​

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-anthropic</artifactId>
    <version>0.36.2</version>
</dependency>

AnthropicChatModel​

AnthropicChatModel model = AnthropicChatModel.builder()
    .apiKey(System.getenv("ANTHROPIC_API_KEY"))
    .modelName(CLAUDE_3_5_SONNET_20240620)
    .build();
String answer = model.generate("Say 'Hello World'");
System.out.println(answer);

Customizing AnthropicChatModel​

AnthropicChatModel model = AnthropicChatModel.builder()
    .baseUrl(...)
    .apiKey(...)
    .version(...)
    .beta(...)
    .modelName(...)
    .temperature(...)
    .topP(...)
    .topK(...)
    .maxTokens(...)
    .stopSequences(...)
    .cacheSystemMessages(...)
    .cacheTools(...)
    .timeout(...)
    .maxRetries(...)
    .logRequests(...)
    .logResponses(...)
    .build();

See the description of some of the parameters above here.

AnthropicStreamingChatModel​

AnthropicStreamingChatModel model = AnthropicStreamingChatModel.builder()
    .apiKey(System.getenv("ANTHROPIC_API_KEY"))
    .modelName(CLAUDE_3_5_SONNET_20240620)
    .build();

model.generate("Say 'Hello World'", new StreamingResponseHandler<AiMessage>() {

    @Override
    public void onNext(String token) {
        // this method is called when a new token is available
    }

    @Override
    public void onComplete(Response<AiMessage> response) {
        // this method is called when the model has completed responding
    }

    @Override
    public void onError(Throwable error) {
        // this method is called when an error occurs
    }
});

Customizing AnthropicStreamingChatModel​

Identical to the AnthropicChatModel, see above.

Tools​

Anthropic supports tools in both streaming and non-streaming mode.

Anthropic documentation on tools can be found here.

Caching​

AnthropicChatModel and AnthropicStreamingChatModel support caching of system messages and tools. Caching is disabled by default. It can be enabled by setting the cacheSystemMessages and cacheTools parameters, respectively.

When enabled,cache_control blocks will be added to all system messages and tools respectively.

To use caching, please set beta("prompt-caching-2024-07-31").

AnthropicChatModel and AnthropicStreamingChatModel return AnthropicTokenUsage in response which contains cacheCreationInputTokens and cacheReadInputTokens.

More info on caching can be found here.

Quarkus​

See more details here.

Spring Boot​

Import Spring Boot starter for Anthropic:

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-anthropic-spring-boot-starter</artifactId>
    <version>0.36.2</version>
</dependency>

Configure AnthropicChatModel bean:

langchain4j.anthropic.chat-model.api-key = ${ANTHROPIC_API_KEY}

Configure AnthropicStreamingChatModel bean:

langchain4j.anthropic.streaming-chat-model.api-key = ${ANTHROPIC_API_KEY}

Examples​