Skip to main content

Anthropic

Maven Dependency

<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-anthropic</artifactId>
<version>1.3.0</version>
</dependency>

AnthropicChatModel

AnthropicChatModel model = AnthropicChatModel.builder()
.apiKey(System.getenv("ANTHROPIC_API_KEY"))
.modelName(CLAUDE_3_5_SONNET_20240620)
.build();
String answer = model.chat("Say 'Hello World'");
System.out.println(answer);

Customizing AnthropicChatModel

AnthropicChatModel model = AnthropicChatModel.builder()
.httpClientBuilder(...)
.baseUrl(...)
.apiKey(...)
.version(...)
.beta(...)
.modelName(...)
.temperature(...)
.topP(...)
.topK(...)
.maxTokens(...)
.stopSequences(...)
.toolSpecifications(...)
.toolChoice(...)
.cacheSystemMessages(...)
.cacheTools(...)
.thinkingType(...)
.thinkingBudgetTokens(...)
.returnThinking(...)
.sendThinking(...)
.timeout(...)
.maxRetries(...)
.logRequests(...)
.logResponses(...)
.listeners(...)
.defaultRequestParameters(...)
.build();

See the description of some of the parameters above here.

AnthropicStreamingChatModel

AnthropicStreamingChatModel model = AnthropicStreamingChatModel.builder()
.apiKey(System.getenv("ANTHROPIC_API_KEY"))
.modelName(CLAUDE_3_5_SONNET_20240620)
.build();

model.chat("Say 'Hello World'", new StreamingChatResponseHandler() {

@Override
public void onPartialResponse(String partialResponse) {
// this method is called when a new partial response is available. It can consist of one or more tokens.
}

@Override
public void onCompleteResponse(ChatResponse completeResponse) {
// this method is called when the model has completed responding
}

@Override
public void onError(Throwable error) {
// this method is called when an error occurs
}
});

Customizing AnthropicStreamingChatModel

Identical to the AnthropicChatModel, see above.

Tools

Anthropic supports tools in both streaming and non-streaming mode.

Anthropic documentation on tools can be found here.

Caching

AnthropicChatModel and AnthropicStreamingChatModel support caching of system messages and tools. Caching is disabled by default. It can be enabled by setting the cacheSystemMessages and cacheTools parameters, respectively.

When enabled,cache_control blocks will be added to the last system message and tool, respectively.

To use caching, please set beta("prompt-caching-2024-07-31").

AnthropicChatModel and AnthropicStreamingChatModel return AnthropicTokenUsage in response which contains cacheCreationInputTokens and cacheReadInputTokens.

More info on caching can be found here.

Thinking

Both AnthropicChatModel and AnthropicStreamingChatModel support thinking feature.

It is controlled by the following parameters:

  • thinkingType and thinkingBudgetTokens: enable thinking, see more details here.
  • returnThinking: controls whether to return thinking (if available) inside AiMessage.thinking() and whether to invoke StreamingChatResponseHandler.onPartialThinking() and TokenStream.onPartialThinking() callbacks when using BedrockStreamingChatModel. Disabled by default. If enabled, tinking signatures will also be stored and returned inside the AiMessage.attributes().
  • sendThinking: controls whether to send thinking and signatures stored in AiMessage to the LLM in follow-up requests. Enabled by default.

Here is an example of how to configure thinking:

ChatModel model = AnthropicChatModel.builder()
.apiKey(System.getenv("ANTHROPIC_API_KEY"))
.modelName(CLAUDE_3_7_SONNET_20250219)
.thinkingType("enabled")
.thinkingBudgetTokens(1024)
.maxTokens(1024 + 100)
.returnThinking(true)
.sendThinking(true)
.build();

Quarkus

See more details here.

Spring Boot

Import Spring Boot starter for Anthropic:

<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-anthropic-spring-boot-starter</artifactId>
<version>1.3.0-beta9</version>
</dependency>

Configure AnthropicChatModel bean:

langchain4j.anthropic.chat-model.api-key = ${ANTHROPIC_API_KEY}

Configure AnthropicStreamingChatModel bean:

langchain4j.anthropic.streaming-chat-model.api-key = ${ANTHROPIC_API_KEY}

Examples