Anthropic
Maven Dependency
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-anthropic</artifactId>
<version>1.5.0</version>
</dependency>
AnthropicChatModel
AnthropicChatModel model = AnthropicChatModel.builder()
.apiKey(System.getenv("ANTHROPIC_API_KEY"))
.modelName(CLAUDE_3_5_SONNET_20240620)
.build();
String answer = model.chat("Say 'Hello World'");
System.out.println(answer);
Customizing AnthropicChatModel
AnthropicChatModel model = AnthropicChatModel.builder()
.httpClientBuilder(...)
.baseUrl(...)
.apiKey(...)
.version(...)
.beta(...)
.modelName(...)
.temperature(...)
.topP(...)
.topK(...)
.maxTokens(...)
.stopSequences(...)
.toolSpecifications(...)
.toolChoice(...)
.toolChoiceName(...)
.disableParallelToolUse(...)
.cacheSystemMessages(...)
.cacheTools(...)
.thinkingType(...)
.thinkingBudgetTokens(...)
.returnThinking(...)
.sendThinking(...)
.timeout(...)
.maxRetries(...)
.logRequests(...)
.logResponses(...)
.listeners(...)
.defaultRequestParameters(...)
.userId(...)
.build();
See the description of some of the parameters above here.
AnthropicStreamingChatModel
AnthropicStreamingChatModel model = AnthropicStreamingChatModel.builder()
.apiKey(System.getenv("ANTHROPIC_API_KEY"))
.modelName(CLAUDE_3_5_SONNET_20240620)
.build();
model.chat("Say 'Hello World'", new StreamingChatResponseHandler() {
@Override
public void onPartialResponse(String partialResponse) {
// this method is called when a new partial response is available. It can consist of one or more tokens.
}
@Override
public void onCompleteResponse(ChatResponse completeResponse) {
// this method is called when the model has completed responding
}
@Override
public void onError(Throwable error) {
// this method is called when an error occurs
}
});
Customizing AnthropicStreamingChatModel
Identical to the AnthropicChatModel
, see above.
Tools
Anthropic supports tools in both streaming and non-streaming mode.
Anthropic documentation on tools can be found here.
Tool Choice
Anthropic's tool choice
feature is available for both streaming and non-streaming interactions
by setting toolChoice(ToolChoice)
or toolChoiceName(String)
.
Parallel Tool Use
By default, Anthropic Claude may use multiple tools to answer a user query,
but you can disable parallel tool by setting disableParallelToolUse(true)
.
Caching
AnthropicChatModel
and AnthropicStreamingChatModel
support caching of system messages and tools.
Caching is disabled by default.
It can be enabled by setting the cacheSystemMessages
and cacheTools
parameters, respectively.
When enabled,cache_control
blocks will be added to the last system message and tool, respectively.
To use caching, please set beta("prompt-caching-2024-07-31")
.
AnthropicChatModel
and AnthropicStreamingChatModel
return AnthropicTokenUsage
in response which
contains cacheCreationInputTokens
and cacheReadInputTokens
.
More info on caching can be found here.
Thinking
Both AnthropicChatModel
and AnthropicStreamingChatModel
support
thinking feature.
It is controlled by the following parameters:
thinkingType
andthinkingBudgetTokens
: enable thinking, see more details here.returnThinking
: controls whether to return thinking (if available) insideAiMessage.thinking()
and whether to invokeStreamingChatResponseHandler.onPartialThinking()
andTokenStream.onPartialThinking()
callbacks when usingBedrockStreamingChatModel
. Disabled by default. If enabled, tinking signatures will also be stored and returned inside theAiMessage.attributes()
.sendThinking
: controls whether to send thinking and signatures stored inAiMessage
to the LLM in follow-up requests. Enabled by default.
Here is an example of how to configure thinking:
ChatModel model = AnthropicChatModel.builder()
.apiKey(System.getenv("ANTHROPIC_API_KEY"))
.modelName(CLAUDE_3_7_SONNET_20250219)
.thinkingType("enabled")
.thinkingBudgetTokens(1024)
.maxTokens(1024 + 100)
.returnThinking(true)
.sendThinking(true)
.build();
AnthropicTokenCountEstimator
TokenCountEstimator tokenCountEstimator = AnthropicTokenCountEstimator.builder()
.modelName(CLAUDE_3_OPUS_20240229)
.apiKey(System.getenv("ANTHROPIC_API_KEY"))
.logRequests(true)
.logResponses(true)
.build();
List<ChatMessage> messages = List.of(...);
int tokenCount = tokenCountEstimator.estimateTokenCountInMessages(messages);
Quarkus
See more details here.
Spring Boot
Import Spring Boot starter for Anthropic:
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-anthropic-spring-boot-starter</artifactId>
<version>1.5.0-beta11</version>
</dependency>
Configure AnthropicChatModel
bean:
langchain4j.anthropic.chat-model.api-key = ${ANTHROPIC_API_KEY}
Configure AnthropicStreamingChatModel
bean:
langchain4j.anthropic.streaming-chat-model.api-key = ${ANTHROPIC_API_KEY}