Skip to main content

Amazon Bedrock

Maven Dependency​

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-bedrock</artifactId>
    <version>1.0.0-beta1</version>
</dependency>

AWS credentials​

In order to use Amazon Bedrock models, you need to configure AWS credentials. One of the options is to set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. More information can be found here.

Difference Between InvokeAPI and ConverseAPI​

Amazon Bedrock offers two primary model invocation API operations for inference:

  • Converse – Amazon recommend using the Converse API as it provides consistent API, that works with all Amazon Bedrock models that support messages.
  • InvokeModel – Originally aimed at single calls to obtain a response to a single prompt.

ChatLanguageModel using ConverseAPI​

Guardrails and streaming are not supported by the current implementation.

Supported models and their features can be found here.

Models ids can be found here.

Configuration​

ChatLanguageModel model = BedrockChatModel.builder()
        .modelId("us.amazon.nova-lite-v1:0")
        .maxRetries(...)
        .timeout(...)
        .logRequests(...)
        .logResponses(...)
        .defaultRequestParameters(ChatRequestParameters.builder()
                .topP(...)
                .temperature(...)
                .maxOutputTokens(...)
                .stopSequences(...)
                .toolSpecifications(...)
                .build())
        .build();

Examples​

ChatLanguageModel using InvokeAPI​

AI21 Models​

  • BedrockAI21LabsChatModel

Anthropic Models​

  • BedrockAnthropicMessageChatModel: supports new Messages API
  • BedrockAnthropicCompletionChatModel: supports old Text Completions API
  • BedrockAnthropicStreamingChatModel

Example:

ChatLanguageModel model = BedrockAnthropicMessageChatModel.builder()
.model("anthropic.claude-3-sonnet-20240229-v1:0")
.build();

Cohere Models​

  • BedrockCohereChatModel

Meta Llama Models​

  • BedrockLlamaChatModel

Mistral Models​

  • BedrockMistralAiChatModel

Titan Models​

  • BedrockTitanChatModel
  • BedrockTitanEmbeddingModel

Examples​