Package dev.langchain4j.model.bedrock

package dev.langchain4j.model.bedrock
AWS Bedrock integration for LangChain4j.

Overview

This package provides integration with AWS Bedrock, Amazon's fully managed service for foundation models. It includes chat models, streaming support, and prompt caching.

Key Classes

Prompt Caching

AWS Bedrock supports prompt caching to reduce latency and costs for repeated content. This package provides two approaches:

1. Simple Caching with BedrockCachePointPlacement

Use for automatic cache point placement after system messages, user messages, or tools:

BedrockChatModel model = BedrockChatModel.builder()
    .modelId("anthropic.claude-3-sonnet...")
    .defaultRequestParameters(BedrockChatRequestParameters.builder()
        .promptCaching(BedrockCachePointPlacement.AFTER_SYSTEM)
        .build())
    .build();

2. Granular Caching with BedrockSystemMessage

Use for fine-grained control over which content is cached:

BedrockSystemMessage systemMessage = BedrockSystemMessage.builder()
    .addText("You are an AI assistant.")        // Not cached
    .addTextWithCachePoint("Large examples...")  // Cached
    .addText("User: " + userName)                // Not cached
    .build();

Choosing Between Approaches

  • Use simple caching when all system content is static and should be cached together
  • Use granular caching when you have mixed static/dynamic content within system messages

AWS Bedrock Caching Requirements

  • Minimum tokens: ~1,024 tokens required for caching to activate
  • Cache TTL: 5-minute default, resets on each cache hit
  • Maximum cache points: 4 per request (across all messages)
  • Supported models: Claude 3.x and Amazon Nova models only

BedrockSystemMessage vs SystemMessage

Important: BedrockSystemMessage implements ChatMessage but does NOT extend SystemMessage. This has implications:

  • instanceof SystemMessage will return false
  • SystemMessage.findFirst() will not find BedrockSystemMessage
  • ChatMemory window managers may not recognize it as a system message
  • Serialization with ChatMessageSerializer is not supported

Use BedrockSystemMessage.toSystemMessage() to convert to a core SystemMessage when needed (cache points will be lost).

See Also: